Validate File Upload Request BEFORE File is Uploaded to Jetty

Question

We have a REST service with an endpoint that receives file upload requests. All requests use an authentication token with a 5 minute timeout.

The problem is that Jetty seems to load the entire file BEFORE calling any of our code. This means that we cannot check the authentication token until after the full upload is finished, which causes large files to fail due to auth timeout.

Let's assume that increasing the Auth timeout is not an option. (Paperwork, red-tape, customer sign off, etc).

First Approach: JAX-RS with MultiPart Form Data

Initially we were using JAX-RS to upload the file like this:

@POST
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.MULTIPART_FORM_DATA)
public Response fileUpload (MultipartFormDataInput input) {
    ...
}

Authentication was checked in a Filter:

@Priority(Priorities.AUTHENTICATION)
public class AuthFilter implements Filter {
    @Override
    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) {
        <check token>
        chain.doFilter(request, response);
    }
}

We've been testing this locally using CURL with a rate limit: curl -v --cookie $TOKEN -F "[email protected]" http://localhost:8080/file --limit-rate 10k

We have log messages in all of our code. The CURL command hangs for a while and none of our logs are printed until the very end, long after the token has expired.

We also tried a ContainerRequestFilter and got the exact same results. Even with the PreMatching annotation.

@Priority(Priorities.AUTHENTICATION)
@PreMatching
public class AuthFilter implements ContainerRequestFilter {
    @Override
    public void filter(ContainerRequestContext requestContext) {
        <check token>
    }
}

To clarify, this approach works fine in most cases. It only has this problem with large files or the rate limiting in CURL.

Second Approach: Using a Servlet

Edit, removed the Apache FileUpload library

We've tried rewriting the endpoint to use a Servlet in hopes that it would allow us to receive the raw request before the file is buffered.

@WebServlet("file")
public class UploadServlet extends HttpServlet {    
    protected void doPost(HttpServletRequest request, HttpServletResponse response) {
        print("Upload started");

       print("Upload finished");
    }
}

However, We're still getting the same results. The log messages from the servlet do not print until after the CURL command is done uploading the file.

Both of these CURL commands have the same results. They hang for a while with no log messages, then both of the Servlet logs appear and the command exits.

curl -v --cookie $TOKEN -F "[email protected]" http://localhost:8080/file --limit-rate 10k

curl -v --cookie $TOKEN -H "Content-Type: application/octet-stream" http://localhost:8080/file --limit-rate 10k --data-binary "@BigFile.zip"

The expectation is that the Servlet logs would appear as soon as the CURL command is started. This would allow us to verify the TOKEN first, then load the file.

Any pointers are appreciated.

Answer 1

I can answer about Jetty behavior, but not your specific REST library.

Jetty is doing exactly what the REST library is telling it to do.

You have declared in your REST library ...

@Consumes(MediaType.MULTIPART_FORM_DATA)
public Response fileUpload (MultipartFormDataInput input) {

That is telling Jetty to load the entire multipart/form-data request body into an object called MultipartFormDataInput and then once it's entirely sent, give that object to your endpoint.

The thing is, the REST library called the Servlet API to access that MultiPart information (eg: HttpServletRequest.getParameter or HttpServletRequest.getPart/getParts) and that caused the read of the entire multipart form based on the REST library configuration of the javax.servlet.MultipartConfigElement.

As to why your REST authentication doesn't trigger, that's outside of what I can help with.

Finally, commons-fileupload should not be used in modern Servlet containers.

See past answers about that:

• How can I upload files to a server using JSP/Servlet?
• Jetty 11 and commons-fileupload

Answer 2

We discovered that our Kong gateway service was actually the culprit. It was buffering the entire file prior to calling the REST endpoint.

There were messages in the Kong logs like this:

[warn] 87400#0: *2723 a client request body is buffered to a temporary file /usr/local/kong/client_body_temp/0000000001

After adding this to our custom-nginx.conf, both of the approaches from the original question work correctly with no modifications.

proxy_request_buffering  off;