Actual maximum concurrent tasks of Parallel.ForEachAsync

Question

I would expect this code to take 1 second to execute:

public async void Test()
{
    DateTime start = DateTime.Now;
    await Parallel.ForEachAsync(new int[1000], new ParallelOptions { MaxDegreeOfParallelism = 1000 }, async (i, token) =>
    {
        Thread.Sleep(1000);
    });
    Console.WriteLine("End program: " + (DateTime.Now - start).Seconds + " seconds elapsed.");
}

Instead, it takes 37 seconds on my pc (i7-9700 8-core 8-thread):

End program: 37 seconds elapsed.

I am generating 1000 tasks with MaxDegreeOfParallelism = 1000....why don't they all run simultaneously?

Answer 1

The Parallel.ForEachAsync method invokes the asynchronous body delegate on ThreadPool threads. Usually this delegate returns a ValueTask quickly, but in your case this is not what happens, because your delegate is not really asynchronous:

async (i, token) => Thread.Sleep(1000);

You are probably getting here a compiler warning, about an async method lacking an await operator. Nevertheless giving a mixed sync/async workload to the Parallel.ForEachAsync method is OK. This method is designed to handle any kind of workload. But if the workload is mostly synchronous, the result might be a saturated ThreadPool.

The ThreadPool is said to be saturated when it has already created the number of threads specified by the SetMinThreads method, which by default is equal to Environment.ProcessorCount, and there is more demand for work to be done. In this case the ThreadPool switches to a conservative algorithm that creates one new thread every second (as of .NET 6). This behavior is not documented precisely, and might change in future .NET versions.

In order to get the behavior that you want, which is to run the delegate for all 1000 inputs in parallel, you'll have to increase the number of threads that the ThreadPool creates instantly on demand:

ThreadPool.SetMinThreads(1000, 1000); // At the start of the program

Some would say that after doing so you won't have a thread pool any more, since a thread pool is meant to be a small pool of reusable threads. But if you don't care about the semantics and just want to get the job done, whatever the consequences are regarding memory consumption and overhead at the operating system level, that's the easiest way to solve your problem.

Answer 2

I do not know the exact implementation of ForEachAsync, but I assume that they use Tasks, not Threads.

When you use 1000 Tasks to run 1000 CPU bound operations, you are not actually creating 1000 Threads, you are just asking a handful of ThreadPool Threads to run those operations. Those Threads are blocked by the Sleep calls, so most of the Tasks are queued up before they can start execution.

This is exactly why it is a horrible idea to call Thread.Sleep in a Task, or in async contexts in general. If you edit your code to wait asynchronously instead of synchronously, the time elapsed will probably be much closer to a second.

await Parallel.ForEachAsync(new int[1000], new ParallelOptions { MaxDegreeOfParallelism = 1000 }, async (i, token) =>
{
    await Task.Delay(1000);
});