Use duration variable as label selector

Question

I have a variable latency_target = 5s. How use it in a PromQL query as a histogram label selector:

sum(rate(histogram{le="$latency_target"}[$interval]))

This does not work, dues to s suffix. I tried to use duration function, but it does not work