How can I fix ConnectTimeout exceptions from within FastAPI

Question

I am looking to create a server that takes in a request, does some processing, and forwards the request to another endpoint. I seem to be running into an issue at higher concurrency where my client.post is causing a httpx.ConnectTimeout exception.

I haven't completely ruled out the possibility of an issue with the endpoint(I am currently working with them to debug anything that might be on their end), but I'm trying to figure out if there is something wrong on my end or if there are any glaring inefficiencies I can improve upon.

I am running this in ECS, currently on a cluster where tasks have 4 vCPUs. I am using the docker image uvicorn-gunicorn-fastapi(https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker). Currently all default settings minus the bind/port/logging. Here is a minimal code example:

import httpx
from fastapi import FastAPI, Request, Response

app = FastAPI()

def process_request(path, request):
    #Process Request Here

def create_headers(path):
    #Create headers here

@app.get('/')
async def root(path: str, request: Request):
    endpoint = 'https://endpoint.com/'
    querystring = 'path=' + path
    data = process_request(request, path, request)
    headers = create_headers(request)
    async with httpx.AsyncClient() as client:
        await client.post(endpoint + "?" + querystring, data=data, headers=headers)
    return Response(status_code=200)

Answer 1

Could be that the server on the other side is taking too much and the connection simply times out because httpx doesn't give enough time to the other endpoint to complete the request?

If yes, you could try disabling timeout or increase the limit (which I suggest over disabling).

See https://www.python-httpx.org/quickstart/#timeouts