The application is a asp.net Core WebAPI. We are using jmeter with taurus to run the tests on an agent in azure devops. When load testing the API we see spikes in response time even if the endpoint returns OK immediately on the first line of code.
The spikes start happening at ~20 hits per second.
What we have tried:
- Scaling up the app service (higher tier plan) and adding multiple instances.
- Configuring the thread pool according to recommendations.
Test result:
When we reach ~20 hit/s the response time jumps from 50ms to almost 20 seconds.
It's hard to tell from the graph above. I would check out Diagnostics and solve problems blade and go through memory and CPU usage. I also suggest enabling application insights to get more details on which API methods are showing increased response times. Furthermore, since you are running a load test, you can also collect a trace from the diagnostics blade and analyze the results.