Counting message (amount and type) per minutes in prometheus

97 views Asked by At

I am trying to fine counting how many messages are received on my server (a custom UDP socket server).

I made a Counter (in python) that just works fine (I can see the right value in Grafana): I can see how many messages are received for minute.

Now I want to know how many clients are active for minute... meaning: in the last minute I've received 200 message by 30 different clients. Since it is a UDP socket I do not have a list of active clients, but I can see the client "ID" from the message payload. The initial idea was to use labels:

c = Counter('my_requests_total', 'Requests')
c.labels(client='id_4510').inc()

but the server can handle thousands of client per minutes and the prometeus docs state:

CAUTION: Remember that every unique combination of key-value label pairs represents a new time series, which can dramatically increase the amount of data stored. Do not use labels to store dimensions with high cardinality (many different label values), such as user IDs, email addresses, or other unbounded sets of values.

How can I do? Shall I use a Gauge and compute the value on server on my own?

1

There are 1 answers

0
markalex On

If all you want is to have info on how many active users you have at the moment - counting on server side would be way more frugally. But in this case you will be bound to algorithm of detection implemented by server at the time.

On the other hand, if you need (or will need in foreseeable future) some kind of elaborate analysis, it would be easier to expose metrics with client label. Additionally, if you'll have a couple of thousand different labels for metric, while generally not advisable, should not have great impact on Prometheus, other than rather than significant disk usage increase.