Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve underlying filesystem handling #139

Open
evillemez opened this issue Aug 28, 2013 · 1 comment
Open

Improve underlying filesystem handling #139

evillemez opened this issue Aug 28, 2013 · 1 comment
Milestone

Comments

@evillemez
Copy link
Contributor

The ideal solution would allow declaring multiple filesystems, and easy migrating of files between filesystem implementations. I see this as happening like so:

ResourceFileManager - takes several "tagged" implementations of file systems, extra methods for migrating specific files across systems, and migrating entire resources.
GaufretteFilesystem - Implements FilesystemInterface for any Gaufrette instance.

I would use https://github.com/KnpLabs/KnpGaufretteBundle to configure multiple storage locations, and configure them to power multiple instances of FilesystemManager.

The end result would let configuration be declared for separate pools of files, for example:

  • upload - shared nfs mount on app server
  • transcoding - probably same location, visible to transcoding servers
  • web - amazon s3
  • cdn - some cdn service

This would let me implement an interface such as:

$resourceFileManager->registerFilesystem('upload', $gaufretteLocalStorage);
$resourceFileManager->registerFilesystem('web', $gaufretteAwsS3Storage);

$resourceFileManager->migrateResource('web', $resource);
$resourceFileManager->migrateFile('transcoding', $resource->content->files[0]);

In dev and testing, all of these can be overridden to simply point to different directories on a local filesystem.

Managing what files belong where, and when they get moved would be fairly straight forward:

  • on upload, file is moved to upload storage
  • before transcode, file is moved from upload to transcode, and new files are created in transcode storage
  • if transcoding is successful, a listener will migrate files to the web storage pool

A few things may need to be refactored for this to work well, but none of them major. Rather than storing an internalUri for files (which assumed they were on local storage, the database should also store a boolean for managed, and internalUri will need to support multiple "schemes" that the manager understands. If the file reference in the database is "managed", then the ResourceFileManager should know how to find it, if not, we must assume it's a reference to a file not under the systems control.

@evillemez
Copy link
Contributor Author

Just an update - if this ever does get implemented or addressed, use Flysystem instead of Gaufrette. It's better and more active:

http://flysystem.thephpleague.com

Aside from the underlying library, the concept remains basically the same, but Flysystem already does a lot of it. Flysystem also allows registering custom schemes, so storing the full scheme urls, such as web:///path/to/file.mp3 or cdn:///path/to/file.mp3 would probably be a viable solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant