http REST consumer timeout

Question

I implemented a https/REST provider in node.js using express. The function is calling a webservice, transforming/enhancing data and returning transformed data as csv using response. Execution time of one get request is between 4 minutes 30 seconds and 5 minutes. I want to test the implementation by calling the url. Problem:

• execution in google chrome fails since it runs to long. No option to increase the time out value.

• execution in modzilla firefox: network.http.response.timeout changed. Now the request is executed over and over again. Looks like the response is ignored completely.

• execution in postman: changed settings->general->XHR timeout in ms(...) .
  Nevertheless execution stops every time after the same amount of seconds with
  message: "Could not get any response" .

My question: which tool(s) can I use for reliable testing of long running http REST requests?

Answer 1

curl has a --max-time in seconds setting which should do what you want.

curl -m 330 http://you.url

But it might be worth creating a background job and polling for completion of the background job instead. HTTP isn't best suited to long running tasks.

Answer 2

I suggest you to use Socket IO to async response with pub/sub when the csv file is ready In the client send the request and put a timeout of 6 minutes for example, the server in the request return an ack to confirm the file process start, when the file is ready, return with Socket IO the file, Socket IO can be integrated with express

http://socket.io/

Answer 3

Do you have control over the server? If so, you should alter how it operates. Instead of the initial request expecting a response containing the answer, your API should emit a token (a URI) from where the status of the operation can be obtained. The status will either be "in progress" or "completed; here's your answer: ..."

You make the problem (the long-running operation) into its own first-class entity on your server.