We have a bunch of SaaS clients. Our SaaS client websites are on IIS on a Windows Server 2019. I have setup a spree commerce backend (hosted using Puma on a debian server) for our SaaS clients. The way it is set up right now is that the client frontends send a request to the IIS server that then gets reverse proxied to the spree backend.
Sometimes (about 10%-15% of the requests), when the frontend makes the request to the IIS server, it responds with 502.3 bad gateway, but it works if I refresh.
Initially I thought the issue was that the Spree backend wasn't responding. But checking the logs for the Puma server, it looks like the request didn't reach the Puma server.
I tried using Failed Request Tracing on IIS and also checked HTTPERR logs. The FRT logs mention ARR_WEBFARM_NOT_ROUTED.
FailedRequestTracing logs:
Looking at HTTPERR logs, it says Connection_Dropped. Here's a snippet of the HTTPERR logs:
#Software: Microsoft HTTP API 2.0
#Version: 1.0
#Date: 2023-12-29 09:26:57
#Fields: date time c-ip c-port s-ip s-port cs-version cs-method cs-uri streamid sc-status s-siteid s-reason s-queuename
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 POST /api/v2/storefront/cart 687 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/products?include=primary_variant%2Cimages%2Cdefault_variant&page=1&per_page=12 703 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/menus?locale=en&filter%5Blocation%5D=header&include=menu_items%2Cmenu_items.linked_resource%2Cmenu_items.icon 699 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/menus?locale=en&filter%5Blocation%5D=footer&include=menu_items%2Cmenu_items.linked_resource%2Cmenu_items.icon 701 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/cms_pages?filter%5Btype%5D=home&include=cms_sections.linked_resource 707 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/taxons?filter%5Broots%5D=true&include=parent%2Ctaxonomy%2Cchildren%2Cimage 695 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/products?include=primary_variant%2Cimages%2Cdefault_variant&page=1&per_page=12 693 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/cart?include=line_items%2Cvariants%2Cvariants.images%2Cbilling_address%2Cshipping_address%2Cuser%2Cpayments%2Cshipments%2Cpromotions 697 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:50:58 209.115.235.218 64176 10.50.7.10 443 HTTP/2 POST /api/v2/storefront/cart 711 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:51:28 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/taxons/2?include=products&fields%5Bproduct%5D=id 683 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:51:28 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/taxons/2?include=products&fields%5Bproduct%5D=id 747 - 7 Connection_Dropped CANbearhouse
2024-01-03 21:52:53 209.115.235.218 64176 10.50.7.10 443 HTTP/2 GET /api/v2/storefront/products?include=primary_variant%2Cimages%2Cdefault_variant&page=1&per_page=12 763 - 7 Connection_Dropped CANbearhouse
How do I go about fixing this? Thanks.