No threads to handle Socket.AsyncReceive because of concurrent requests to WCF service

Question

I have a WCF service hosted in IIS that on request does a request to another server using asynchronous socket requests. The request has to be fast (5 seconds) or it times out. When a lot of users are sending requests at the same time (I guess about 10) then what happens is that socket requests to another server are not handled within those 5 seconds. The logs on another server say that the request is received and handled almost immediately and the response is sent back, but the async event is not fired before 5 - 10 seconds causing the wcf service to have a timeout (note this is not a timeout of the wcf itself, but an internal timeout on a call to another server). We think that that thread pool is either too slow to spawn new threads or has some reasons not to do that, which leads to the situation when there are no threads available to pick up the async receive of the socket. As far as I understand both WCF and async socket handlers are using ThreadPool for that so they really fight for the same resource. Did someone encounter similar situation? What is the best way of handling it?

I see two easy ways out:

1) Limit the number of maximum concurrent requests at the wcf service to 5 or something which may cause it to be slower as some requests will have to wait in the queue.

2) Increase the number of MinSize in the ThreadPool I would be glad for any suggestions on the matter.

P.S. The processor has 8 cores, meaning default minsize of threadpool is 8, so I start seeing the problem when more than 8 concurrent requests are happening.

Answer 1

If you are blocking while waiting for async IO, you are creating a situation that is guaranteed to create ThreadPool thread starvation.

Eight requests come in and use all threads in the ThreadPool. They issue an asynchronous IO request, all is good. IO requests are answered, and async callbacks are queued to the ThreadPool, but now all threads in the TheadPool are tied-up waiting for the operation that we've just queued up to a blocked queue. Now after a while, the ThreadPool notices that the queues aren't shrinking and throws a thread at the problem. Slowly the spare thread in the system relieves the deadlock, maybe the ThreadPool spins up another thread if the queue isn't clearing down quickly enough. The important thing is that it isn't in any kind of rush to deal with this increased workload, and optimistically uses as few thread as it can get away with to clear the backlog. Not good. Loads of latency.

You need to rewrite your service contracts to return Task<T> instead of plain'old T. Your clients won't even notice, but you'll be able to run thousands instead of 8 concurrent clients without the slightest strain (assuming the actual workload is trivial) by adopting async/await.

If this isn't an option, consider performing your IO using synchronous methods, because currently, the thread you're running on and the thread that you're depending on in order to proceed are both running in the same constrained ThreadPool. With sync IO, you'll remove a large part of the problem, although it will still (IMO) be severely hobbled by not writing an async service method.

EDIT

Seeing as you on an older version of .Net, you probably can't use async/await. There are a couple of other options available for writing async services, documented here. IIRC, although you change the contract to implement these methods, the endpoints w.r.t. the client stay the same as long as you follow the conventions laid out in the documentation.