I'm using Prometheus to show graphs in grafana.
One of my metrics is a histogram called api_duration_milliseconds which I use like this:
sum(rate(api_duration_milliseconds_sum[5m])) by (url) / sum(rate(api_duration_milliseconds_count[5m])) by (url)
but the rate function may return 0 when my observations have the same value for a while. How can I resolve this? For example, if I just started sending the metrics and only has 1 observation - 20ms. I would want to see 20ms and not 0.
One thing that may help is to first aggregate the data from all machines that send the metrics (between all of them I have less chance of having the same value) and then applying the rate method. but I was not sure how to do this.