Kiali is unable to connect to Prometheus deployed in another namespace

Question

I have Kiali dashboard installed in the istio-system namespace. Then in the kiali configMap in the istio-system namespace, I have added the dns for prometheus service (e.g. http://custom-prometheus.monitoring.svc.cluster.local:9090).

external_services:
  prometheus
    url: "http://custom-prometheus.monitoring.svc.cluster.local:9090"

I am getting this error from the Kiali dashboard UI:

Could not fetch health: Error while fetching app health: client_error: client error: 404

On checking the kiali logs, also, I see the following error:

ERR Failed to fetch Prometheus configuration: client_error: client error: 404
ERR Failed to fetch Prometheus flags: client_error: client error: 404

Note:

• I have checked that my prometheus instance is working and I am able to see the dashboard when I port-forward the service.
• I have also tried using pod IP for prometheus and still getting the above error.
• Istio version: 1.17.1, Kiali version: 1.66.1