Azure App Service - CPU Percentage of instance vs plan

Question

I am running a load test against my Azure Web App P3 pricing tier. We have the following auto scale out strategy -

Min Instance 5 and Max instance 20, Increase by 1 Instance if CPU Percentage (Max) goes more than 85%, Decrease by 1 Instance if CPU Percentage Average goes below than 50%

Right now, it is running with 5 instances. If I go to Applications Insights, 'Live Metrics Stream' pane of all available instances, it shows that CPU usage is around 75% (average) in all the 5 instances. In fact, some of the instances are nearing 85%.

Whereas, if I turn to CPU usage chart on the App Service Plan level (I have only one app running under the plan), it shows only 20%.

How do we reconcile these two conflicting stats?

Answer 1

According to your description, I tried to run the load test against my Azure Web App. Here are the detailed metrics from Application Insights and App Service Plan, you could refer to it.

Application Insight > Metrics Explorer

Time range within an hour

PROCESS CPU: The percentage of elapsed time of all process threads used by the processor to execute the applications process.This metric is an average. Each point on the chart represents 1 minute of data.

App Service Plan > Monitoring

CPU percentage on the other hand is useful for apps hosted in basic, standard and premium plans since they can be scaled out and this metric is a good indication of the overall usage across all instances.

Note: Since the Live Metrics Stream provides you the metrics within the latest 60 seconds. Please try to compare the metrics from App Service Plan and Application Insights within the same Time range.

Per my understanding, the trend of metrics from Service Plan and Application Insights is approximately the same. For auto-scaling, I assumed that you'd better refer to the metrics from the App Service Plan level. Here is a screenshot about the history metrics when you adding the rule for auto-scaling, which matches the metrics from "App Service Plan > Monitoring" with the same time range.

Metrics when adding the scale rule:

App Service Plan > Monitoring

Answer 2

What is shown in 'Live Metrics Stream' is the CPU usage of w3wp process, whereas what is shown in App Service Plan level is the total Machine CPU usage. The former is not normalized for take into account the number of logical processors - so you need to divide it by number of cores to get the normalized percentage. Even after this, 'Live Metrics Stream' metric can be lower than AppServicePlan metric as the former only shows w3wp usage and the latter shows total machine cpu usage.