What is the proper way to update Prometheus counters from goroutine?

189 views Asked by At

I have a function, which handles some events:

func (h *Handler) OnEvent(e *Event) {
    log.Printf("%s, %s", e.Label, e.Type)
}

I want to create a Prometheus counter which will tick on each event. There are two caveats:

  1. OnEvent runs in goroutine
  2. e.Label and e.Type could be different in different periods of time, so I can't pre-register all counters on startup.

My main question is how to register Prometheus counters safely? If I do this inside OnEvent

func (h *Handler) OnEvent(e *Event) {
    log.Printf("%s, %s", e.Label, e.Type)
    c := prometheus.NewCounter(prometheus.CounterOpts{
        Name: "example",
        Help: "Example.",
        ConstLabels: map[string]string{
            "label": e.Label,
            "type":  e.Type,
        },
    })
    prometheus.MustRegister(c)
    c.Inc()
    
    pusher := getPusher()
    if err := pusher. //push new counter value to Pushgateway
        Collector(c).
        Push(); err != nil {
        fmt.Println(err)
}

I will get panic on MustRegister on 2nd event with identical label and type. There is no way to "retrieve" an existing counter, so I want to ask, how should I register all counters and update them.

2

There are 2 answers

0
Trock On

Example:

package main

import (
    "github.com/prometheus/client_golang/prometheus"
)

var c = prometheus.NewCounterVec(prometheus.CounterOpts{
    Name: "example",
    Help: "Example.",
}, []string{"label", "type"})

type Handler struct {
}

type Event struct {
    Label string
    Type  string
}

func (h *Handler) OnEvent(e *Event) {
    c.WithLabelValues(e.Label, e.Type).Inc()
}

func main() {
    prometheus.MustRegister(c)

    (&Handler{}).OnEvent(&Event{
        Label: "a",
        Type:  "1",
    })

    (&Handler{}).OnEvent(&Event{
        Label: "b",
        Type:  "2",
    })
}
0
hmmftg Stands with Palestine On

in your case it's better to have a single guage named event which has two labels: label and type, on each event you will SetGaugeValue for both labels:

// AddGauge adds a custom gauge and registers it. 
func AddGauge(
    namespace, subsystem, name, help string, 
    labels []string) *prometheus.GaugeVec
 {
    reg := prometheus.NewRegistry()
    g := prometheus.NewGaugeVec(prometheus.GaugeOpts{
        Namespace: namespace,
        Subsystem: subsystem,
        Name:      name,
        Help:      help,
    },
        labels)
    reg.MustRegister(g)
    return g
}
// on app start up phase: (save g in a proper interface, Handler in your case)
g:=AddGauge("namespace","subsystem","event", "Events.", []string{"label","type"})
// OnEvent
func (h *Handler) OnEvent(e *Event) {
    log.Printf("%s, %s", e.Label, e.Type)
    h.WithLabelValues(e.Label, e.Type).Inc()
}