Async vs Horizontal scaling

Question

I am working on a .net client which consumes a REST api (JSON). The client is a web application with high traffic and considerable user interaction

When writing the wrappers around the external REST api i am trying to decide

1. If i should make all calls made for the API async? This will be all the way from UI to the API as explained here http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html. This will help me in achieving the desired performance but i will have to figure out a way to handle UI when Tasks are waiting to be completed

2. Or Is it an overkill? And do i just use sync/sequential code? I could still get (some) performance by Horizontally scaling the application?

I am keen to understand what is the preferred way of calling an external REST api from a client (if there is any) and how is the UI handled for those case where people do use async?

Answer 1

So you have about 10 requests per second in a busy period. This by itself does not require asynchronous IO. Assuming 1sec per request that's 10 threads. 10 threads are nothing.

There is one special case, though: What if the backend service you are calling sometimes takes a long time to respond (a bug, overload, index rebuild, ...)? If it takes 30 seconds to respond or timeout that means that 300 requests are in flight. This is too much for the default thread-pool settings. This will effectively shut down the entire app by pool exhaustion until the requests are cleared.

You can do two things:

1. Use async IO for all high-volume actions.
2. Reduce timeouts and have a load-breaker for the number of in-flight requests. Example:

.

SemaphoreSlim sem = new SS(50); //max 50 in-flight

//...

if (!sem.WaitOne(TimeSpan.Zero))
 throw new Exception("Load limit exceeded");

Both are equally safe and well-performing. Do not make the mistake to think that async IO causes your IOs to become faster.

The semaphore solution requires less architectural changes but it requires permission to drop requests. These requests would not have completed anyway with high likelihood.