Unauthorized response when making http calls in parallel

Question

I have code where I loop through some services and call the api for those services. If I run these calls one at a time, I get the response StatusCode of OK (200) for each call. If I change the code to run in parallel, the first call finished gets an OK but the remaining calls all complete with a StatusCode value of Unauthorized (401).

What's weird is that all the calls are actually processed on the server -- even the calls where I get back an Unauthorized response. I don't understand what is happening.

Here is the code I'm running. (Parallel code is currently commented out)

using (HttpClientHandler handler = new HttpClientHandler())
{
    handler.UseDefaultCredentials = true;  // Use Windows Credentials
    using (var client = new HttpClient(handler))
    {
        //Some services can run up to 1.5 minutes - need to ensure we don't time out
        client.Timeout = new TimeSpan(0, 3, 0);

        //Get Available Services
        Console.WriteLine("Retrieving service list ...");
        var result = await client.GetStringAsync(_baseURL);

        var jss = new JavaScriptSerializer();
        List<Service> services = new List<Service>();
        services = jss.Deserialize<List<Service>>(result);
        Console.WriteLine(services.Count.ToString() + " services to run.");

        // ** Non Parallel Code **
        foreach(var service in services)
        {
            Console.WriteLine("Running " + service.Name);
            var _serviceResponse = await client.PostAsync(_baseURL + "/" + service.Id.ToString() + "/run", null);
            Console.WriteLine(service.Name + ": Response = " + _serviceResponse.StatusCode + "  " + _serviceResponse.ReasonPhrase);
        }

        //  ** Parallel Code **
        //await Task.WhenAll(services.Select(async s => {
        //        Console.WriteLine("Running " + s.Name);
        //        var _serviceResponse = await client.PostAsync(_baseURL + "/" + s.Id.ToString() + "/run", null);
        //        Console.WriteLine(s.Name + ": Response = " + _serviceResponse.StatusCode + "  " + _serviceResponse.ReasonPhrase);
        //}));
    }
}

}

From what I've read, HttpClient is supposed to be able to process simultaneous calls without problem when using Async (which I'm doing). So I'm curious why it behaves differently with this code.

Answer 1

It turns out the service I was calling didn't like that many requests coming that quickly.

I had to prime the online list with an upload and then use my code to only upload a small number of additions or deletions.

Answer 2

Is it possible the server is applying some kind of rate limiting to client requests from the same IP/User, and returning 401 when the max is exceeded?