How do you restrict large file uploads in wsgi?

Question

I'm trying to get an understanding of the best way of handling file uploads safely in a wsgi app. It seems a lot of solutions involve using FieldStorage from the cgi module to parse form data. From what I understand about FieldStorage it performs a bit of 'magic' behind the scenes by streaming data into a tempfile.

What I'm not 100% clear on is how to restrict a request containing a file greater than a specified amount (say 10MB). If someone uploads a file which is several GB in size you obviously want to block the request before it chews through your server's disk space right?

What is the best way to restrict file uploads in a wsgi application?

Answer 1

It would depend on your front-end server. If it has any configuration to block big request even before it goes into your app, use it.

If you want to block this with your code I see two approaches:

• Look ate the Content-Length HTTP Header. If it's bigger than you can handle, deny the request right away.
• Don't trust the headers and start reading the request body, until you reach your limit. Note that this is not a very clever way, but could work. =)

Trusting the HTTP header could lead you to some problems. Supose some one send a request with a Content-Length: 1024 but sends a 1GB request body. If your front-end server trusts the header, it will start do read this request and would find out later that the request body is actually much bigger that it should be. This situation could still fill your server disk, even being a request that "passes" the "too big check".

Although this could happen, I think trusting the Header would be a good start point.

Answer 2

You could use the features of the HTTP server you probably have in front of your WSGI application. For example lighttpd has many options for traffic shaping.