Proxy gives 502 Bad Gateway error when trying to download file from express app behind proxy

Question

I am using ISAM proxy app (let's call it Proxy-APP) powered by Express. ISAM uses reverse proxy concept to redirect the traffic to backend express app (call it App).

Problem:

We are having a download functionality written in App. Similar to below :-

router.get('/fileDownload', function (req, res, next) {
  res.setHeader('Content-disposition', 'attachment;filename=' + req.query.fileName);
  request(req.query.url).pipe(res);
});

Above code make request to an external URL (contains some actual file) and then pipe the response. Intermittent we are getting 502 Bad Gateway in Proxy-App logs.

There is pattern when we get 502 error. If Proxy-App doesn't receive the response from App in 2 minutes, It gives the 502 error.

How can we write above code more effectively, so that when the response doesn't come from req.query.url we can handle the error.

OR

What can we modify in Proxy-App so that we can increase the 2 minutes timeout. Below is the context and options of http-proxy-middleware.

  var context = '/';
  var options = {
    target: TARGET,
    changeOrigin: true,
    logLevel: 'info',
    agent: https.globalAgent,
    timeout: 3600 * 1000
  };

  var proxy = proxyMiddleware(context, options);
  app.use(proxy);

Note: Above timeout is not having any effect on the solving the problem.

Edit: request a module from node.js.

Answer 1

First of all there was no issue from the ISAM. They responded back that 502 Bad Gateway error is coming from the backened server. i.e. our App.

First problem was with default timeout of 2 mins. which is hardcoded in node see link. This blog help me to identify this.

But after raising the setTimeout for server/request/response, I was getting the file in longer time. Basically it was not timing out. (No 502 error).

Still this didn't solve my problem. I wanted it fast, not after some time. Googling more gave me a node module called hyperquest. which is design for the similar problem. It has mentioned that request module has some limitation for stream the data over http.