I have a pipeline where a request hits one server, and that server calls another server and that one executes a job for two seconds, then should return to the main server for it to do some minor computation, then return to the client. The problem is that my current setup blocks if number of concurrent requests > number of workers, and I don't know how to use Python threading to make it async. Any ideas on how to implement this?
Main server -> Outside server -> 2 seconds -> Main server
:Edit
The line that takes 2 seconds is the one with the "find_most_similar_overall(image_name, classifier, labels)" call. That function takes 2 seconds, which means that the worker stops right there.
@app.route("/shoes/<_id>") def classify_shoe(_id):
if request.method == 'GET':
unique_user = request.cookies.get('uniqueuser')
shoe = Shoe.query.filter_by(id = _id)
if shoe.count() is not 0:
shoe = shoe.first()
image_name = shoe.image_name
shoes,category = find_most_similar_overall(image_name, classifier, labels)
return render_template('category.html', same_shoes = similar_shoes,shoes=shoes,main_shoe=shoe,category=category, categories=shoe_categories)