BenchmarkDotNet GlobalSetup does not handle async properly

427 views Asked by At

I have a benchmark class with code similar to the following:

[GlobalSetup]
public async Task Setup()
{
    await Task.Delay(1000 * 10);

    Console.WriteLine("Got here...");
}

[Benchmark]
public async Task Benchmark()
{
   await OtherCode();
}

When I run this, OtherCode is ran before the GlobalSetup finishes entirely. Is this a bug in BenchmarkDotNet, or am I likely doing something incorrectly?

I've attempted to search online, but most of my results brought up the same handful of articles that didn't have any useful info.

Edit:

https://github.com/dotnet/BenchmarkDotNet/issues/1738

This link shows a similar problem and it doesn't look like it ever got solved. Has anyone dealt with a problem similar to this, and if so, what was your work-around / solution?

Edit 2:

With my specific setup, I was able to get it working, though it does seem a bit hacky. The code that I had in the GlobalSetup I instead calculated beforehand and stored in a static field. Then I referenced that in the benchmark class.

It's also important to note that I'm running my benchmark class from an xunit test and with InProcessEmitToolchain in the job's config for the runner.

That being said, I'm still curious as to why GlobalSetup doesn't get awaited properly. Is this in the works to be fixed, or is it intentional?

1

There are 1 answers

0
Knight Steele On

https://github.com/dotnet/BenchmarkDotNet/issues/1738#issuecomment-1687832731

As per this issue, there is a pull request up to fix this, though it has been in an open state for quite some time. As the pull request's author mentioned, a temporary solution is to wait synchronously for the result task.GetAwaiter().GetResult() within your GlobalSetup. I believe this makes more sense than my original solution, as it keeps your logic within the benchmark itself, rather than injecting it through a static wrapper.