Azure IoT Device to Cloud, Metrics graph drops to zero at a particular time stamp

Question

I have an Azure IoT device connected to an Azure IoT Hub. The device sends 6 - 7 messages per minute. By looking at the D2C message metrics, I found an outlier, that states that at a specific time, the count of the D2C message was zero (see picture). As the messages are routed to a storage, I can check the storage to see if there are messages missing at that specific time, but the data saved in the storage shows that every message was received correctly at that time. Does any one know how that comes or if the metrics are not that reliable generally? If that's the case, what is the best practice to monitor the IoT Hub message transfer?

IoT Hub D2C Message Metrics

EnqueuedTimeUtc in the storage

Answer 1

For precisely monitor the flow of each message through IoT Hub you will need to Trace Azure IoT device-to-cloud messages with distributed tracing (currently in preview)

• This trace context includes correlation IDs that allow you to correlate events from one component with events from another component
• Automatically log the trace context to Azure Monitor Logs.
• Measure and understand message flow and latency from devices to IoT Hub and routing endpoints.

Ref: https://learn.microsoft.com/en-us/azure/iot-hub/iot-hub-distributed-tracing