My favourite type of plugin is one that has focused single job, requires little configuration and “just works”. I’m not so interested in user facing features, or admin options; I want a plugin to do one thing and do it well.
One such plugin that has been in development for some time here at Human Made is S3 Uploads. This is a single purpose plugin that enables your uploads library to be hosted on Amazon S3 rather than on the same server as WordPress.
Motivation
There are a number of reasons why you would want to store you uploads on Amazon S3 (or any other external storage), to name a few:
- No need for shared filesystems (e.g. NFS) for the
uploadsdirectory in multi-server environments. - Much easier to migrate between hosts as the uploads of a site don’t need to move.
- Reduce load / responsibility of the web server by offloading requests for files.
- Smaller security exploit footprint due to uploaded files being hosted in a non-executable environment.
We primarily use S3 Uploads to solve the network file system problem. Setting up NFS or other systems in a truly distributed way is time consuming and error prone. By storing the files elsewhere (and where more reliable than S3!) it becomes easy to horizontally scale the number of web servers.
Isn’t that slow?
Actually, no. Or rather – not if you are doing things right. WordPress does not need a persistent connection to continually read files from the uploads directory. The only time a file is access is on upload (typically). This makes the latency between the server and Amazon S3 not a large issue, aside from some quirks (looking at you wp_upload_dir) that S3 Uploads provides workarounds for.
S3 Uploads operates at a very low level which allows it to be used in a “drop-in” fashion to seamlessly and stably replace the default WordPress behaviour. This is achieved by using a custom PHP Stream Wrapper in-place of the default file handler stream wrapper for the uploads directory. Underlying functionality from WordPress (and plugins) can continue to do everything they usually do, all the while oblivious to the change in the location of the uploads directory, thanks to the PHP Streams API.
S3 Uploads is meant to be used by developers, rather than users. It has no admin UI and is configured using PHP constants in the wp-config.php. See details on the GitHub readme.
The plugin can migrate your content to Amazon S3, generate a IAM user with the correct privileges for you and provides a suite of wp-cli commands for common tasks such as uploading and listing files on S3.
Usage and Track Record
S3 Uploads has not yet hit version 1.0, however it is quite broadly tested thus far. We use it on around 30 sites, some with 100’s of gigabytes of uploads and millions of page views per month. For stability and performance problems, we should be the first to notice!
Download
The plugin is not available on the WordPress.org plugin repository, this is mainly because it is intended for developer use rather than the user facing nature of the Plugins repository. I see S3 Uploads more like the object-cache.php drop-in for example.
You can download the plugin from GitHub, the currently version is 0.9.0. Feel free to let us know what you think!
Harness the power of plugins and build efficient websites at scale