I have a setup with Prometheus and Thanos in a hub and spoke model, where the spoke Prometheus instances send their metrics to the hub Prometheus via remote write. The hub has a Thanos Sidecar attached, and I use Thanos Querier to query the metrics. While I can see the hub's local metrics in the Thanos Querier, I don't see the metrics from the spoke clusters that were sent to the hub via remote write.
Is this expected behavior for Thanos Querier, and does it only query metrics stored locally on the hub's Prometheus instance, ignoring the metrics received through remote write? How can I configure Thanos or Prometheus to make the remote write metrics visible in Thanos Querier?"