Handling 'no data' in Grafana subtraction query using LogQL with Loki

314 views Asked by At

I'm using Promtail to forward logs to Loki and visualize the data with Grafana. I have a query that calculates the number of devices online without errors. The query works as follows:

(
  (count(count by(system) (count_over_time({job="mrs_error_list"} |~ "" [7d]))))
)
- 
(
  (count(count by(system) (count_over_time({job="mrs_error_list"} |~ "Error" [5m]))))
)

This works perfectly as long as both parts of the subtraction always have data. However, sometimes there are no error data in the last 5 minutes. When this happens, the entire subtraction evaluates to 'no data', but I'd prefer it to evaluate as subtracting zero.

What i tried: I tried to use something like or vector(0), but this doesn't seem to exist yet for LogQL. I attempted to use "Add field from calculation," but this returns the exact same error. I tried to separate the query into two different ones and use the override option "Standard options > no value" so that it defaults to 0, but I can't subtract this value from the first query then.

What i am expecting: I want the query to interpret 'no data' as zero, so the subtraction can still take place and yield a meaningful result in Grafana.

1

There are 1 answers

1
markalex On BEST ANSWER

Premise in your question is incorrect. Loki supports constructions like or vector(0), and it's even used in examples in their documentation.

So your query will look like this:

count(count by(system) (count_over_time({job="mrs_error_list"} |~ "" [7d])))
- 
(
  count(count by(system) (count_over_time({job="mrs_error_list"} |~ "Error" [5m])))
  or vector(0)
)

Alternatively, based on my answer to your previous question, you might count diference instead of subtracting counts:

count(
  count by(system) (count_over_time({job="mrs_error_list"} |~ "Timestamp" [7d]))
  unless
  count by(system) (count_over_time({job="mrs_error_list"} |~ "Timestamp" [1m]))
)

You can see couple live examples of similar queries here.