Serving static files through MongoDB's GridFS

Question

I'm fairly new to Django and I'm trying to deploy a small hobby project on OpenShift. I know that there are certain conventions that recommend against letting Django serve static and media files because it's inefficient. I've also noticed that Django refuses to serve media files when DEBUG is turned off. That's why I'm looking at better ways to serve this content.

Django's documentation endorses CDNs like Amazon S3 as one of the best way to serve static, but as a hobbyist I'd rather stick to freemium solutions for now. I found out that MongoDB - another technology I'm fairly new to - provides GridFS as a storage backend. I can get free MongoDB storage through MongoLab, so this is looking interesting to me.

Would such a construction work in practice or is this crazy talk? If this is feasible, which changes would I need to make to my OpenShift environment and Django settings to let GridFS serve static content? I've seen alternative setups where people use CloudFlare's free CDN to serve their static content, but then I wouldn't be able to upload/access media files from my local development environment.

Answer 1

To make a long story short: it is a bad idea.

Here is why: in order to serve static files, you would first need to process the request, get the data from GridFS, which actually scatters the files in 255k chunks which would have to be collected (depending on the size, of course) and only then they could be returned.

What I tend to do is to use varnish to cache the static files served by the application, be it either django or a Servlet container. This works like this:

1. All requests are sent to varnish, which either serves the requested ressource from cache or hands then to a backend.
2. Varnish uses your django app as a backend. I usually have django run behind an additional lighttpd, though.
3. When a static file is returned by django, varnish puts this file into an in memory caching area. I usually have some 100M allocated for this - sometimes even less. The size of your static files should be well known to you. I tend to do a simple caching configuration like "cache all files ending with .css, .js, .png".
4. All subsequent requests for this static ressource will be served by varnish now - from memory, via the sendfile system call, not even hitting the backend.

All in all: this way, load is taken from your application, latency is reduced, the ressources are delivered lightning fast, it is easy to set up without any programming effort.

Edit: as for an OpenShift environment: simply leave it as is. Serving static files from MongoDB simply does not make sense.