Use PromQL/MetricsQL-Querydata for Grafana only if condition in another metric is met

Question

I have two similar metrics from the same job. It's the probe_success and probe_http_status_code metric of the Prometheus Blackbox Exporter. I'm also using Victoriametrics if that's of any help.

I now want to create a Grafana-panel that displays the probe_http_status_code of various entities but only if they have a probe_success == 0 in the given range.

I tried to receive both metrics at once taking advantage of the union-functionality of MetricsQL. To only receive the labels, that allow to connect both metrics, I'm using label_keep:

(label_keep(probe_success == 0, "instance"), label_keep(probe_http_status_code, "instance"))

Unfortunately the Status codes are discarded, as there are already the values of the probe_success-metric. Is there any way to combine them in a way that all Status Codes which correspond to a probe_success of 0 are discarded? Or maybe the value of probe_success may be transformed to a label to be lateron filtered in grafana itself?

Answer 1

To get list of all the received http codes for target that failed at least once within the range of your dashboard use query

probe_http_status_code and min_over_time(probe_success [$__range] @end() ) == 0

Here probe_http_status_code is returned only if min_over_time(probe_success [$__range] @end() ) == 0 also returned result with same labels. And the latter return result if within values of probe_success there was at least one zero (based on the fact that only possible values are 0 and 1).

Construct probe_success [$__range] @end() return range vector of length $__range (substituted by Grafana with actual range of dashboard) that ends on dashboard end time. @end() is used to pin this range selector to the end of dashboard for all values, otherwise range selector would return range vector relatively to the time of evaluated moment.