I have the following PromQL query to get the CPU percentages of the nodes in my Kubernetes cluster.
100 - (avg by (instance) (rate(node_cpu_seconds_total{mode!="idle|wait|iowait"}[5m])) * 100) > 80
I also have setup the alerts.
Problem I have here is, even though the alerts are triggering, when the CPU is > 80%, when I actually check the nodes with kubectl top nodes or from Azure Portal (as I have an AKS cluster), the nodes' CPU levels are nowhere near 80% (most of the times they are around 20%-30%)
Could someone please help me to adjust this query so I can get the average CPU of each node?