Unable to calculate CPU usage % properly

Question

I am using the script provided here with a small difference as quoted below to fetch the CPU usage of a droplet VM.

snd date as current time in seconds start date as current time - 180 seconds

But still I am not able to fetch the utilization properly. Screenshot of shell script output and CPU utilization graph is provided.

Any idea why am I not getting the actual utilization as per the graph?

Update 1:

#!/bin/bash


TOKEN="myToken"
HOST_ID="myId"

while true
do
# Get the current time in seconds since the epoch
end_time=$(date +%s)

echo "Current time: $end_time"

# Calculate 5 minutes before the current time
start_time=($(date +%s) - 180)

API_ENDPOINT="https://api.digitalocean.com/v2/monitoring/metrics/droplet/cpu?host_id=$HOST_ID&start=$start_time&end=$end_time"

# Get the metrics
RESPONSE=$(curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" "$API_ENDPOINT")

# Parse the metrics
IDLE=$(echo "$RESPONSE" | jq -r '[.data.result[] | select(.metric.mode == "idle") | .values[0][1]] | add')
TOTAL=$(echo "$RESPONSE" | jq -r '[.data.result[] | .values[0][1] | tonumber] | add')

echo "IDLE: $IDLE"
echo "TOTAL: $TOTAL"

USED=$(echo "$TOTAL - $IDLE" | bc)

echo "USED: $USED"

ZERO_CHECK=$(echo "$TOTAL == 0" | bc)
if [ $ZERO_CHECK -eq 1 ]; then
 echo "No change in TOTAL, can't calculate CPU Usage"
else
 CPU_USAGE=$(echo "scale=2; ($USED / $TOTAL) * 100" | bc)
 echo "CPU Usage: $CPU_USAGE%"
 fi
done

Answer 1

I've just tested this with the following modification:

API_ENDPOINT="https://api.digitalocean.com/v2/monitoring/metrics/droplet/cpu?host_id=$HOST_ID&start=$(($(date +%s) - 180))&end=$(date +%s)"

Note the ($(date +%s) - 180) in the start time.

And it seems to be working as expected.

What is the exact change that you made in the script?

Answer 2

I notice in the shell script output that your "Current time" output is increasing by about 30 seconds per iteration - there's no "sleep" or anything else in your script that'd take that kind of time so that means each call to curl is taking about 30 seconds to complete which is a very long time - are you sure it's not timing out and populating RESPONSE with something other than what you expect?

As you can see in your output, the IDLE and TOTAL values are very similar so if those values are wrong then it's either a problem with your curl command or with how you're using jq to parse the curl output. You didn't show us an example of $RESPONSE so we can't tell if there's an issue in that and I'm not very familiar with jq but I notice you have:

IDLE=$(  '[ ... | .values[0][1]] | add' )
TOTAL=$( '[ ... | .values[0][1]  | tonumber] | add' )

i.e. you call tonumber in one case but not the other, I don't know for sure if that's a problem or not but it looks to me like you either need tonumber in both places or in neither of them.

Also, by using CPU_USAGE=$(echo "scale=2; ($USED / $TOTAL) * 100" | bc) with a small value of USED and large value of TOTAL the result of USED / TOTAL is 0 to 2 decimal places before you multiply it by 100 and still get zero. You'd get a more accurate result with (USED * 100) / TOTAL.

I fixed that issue and generally tidied up your script below, try it and post a new question if it still doesn't do what you want but this time only post text, no images, and print the contents of $response and $start_time as well as everything else you're currently printing..

#!/usr/bin/env bash

token='myToken'
host_id='myId'

while true
do
    # Get the current time in seconds since the epoch
    end_time=$(date +%s)

    printf 'Current time: %s\n' "$end_time"

    # Calculate 5 minutes before the current time
    start_time=$(( end_time - 180 ))  # <- actually 3 minutes

    printf -v api_endpoint 'https://api.digitalocean.com/v2/monitoring/metrics/droplet/cpu?host_id=%s&start=%s&end=%s' "$host_id" "$start_time" "$end_time"

    # Get the metrics
    response=$(curl -X GET -H "Content-Type: application/json" -H "Authorization: Bearer $token" "$api_endpoint")

    # Parse the metrics
    idle=$( printf '%s\n' "$response" | jq -r '[.data.result[] | select(.metric.mode == "idle") | .values[0][1]] | add')
    total=$(printf '%s\n' "$response" | jq -r '[.data.result[] | .values[0][1] | tonumber] | add')

    printf 'idle: %s\n' "$idle"
    printf 'total: %s\n' "$total"

    used=$(printf '%s - %s\n' "$total" "$idle" | bc)

    printf 'used: %s\n' "$used"

    zero_check=$(printf '%s == 0\n' "$total" | bc)
    if (( zero_check == 1 )); then
        printf "No change in total, can't calculate CPU Usage\n"
    else
        cpu_usage=$(printf 'scale=2; (%s * 100) / %s\n' "$used" "$total" | bc)
        printf 'CPU Usage: %s\n' "$cpu_usage"
    fi
done