I have several globally distributed tagging server deployments that are deployed in GCP Cloud Run.
The Cloud Run deployments are reachable through serverless network endpoint groups.
The traffic is balanced by a Classic HTTPS Application Load Balancer. I now want to measure the RTT from several source locations (e.g. India, China, etc.) to the tagging servers, means the time the clients browser needs to get a result from the tagging server.
I did use dotcom-monitor.com to measure the latency to the IP of the LB, but this obviously only reflects the response of the LB and does not take into account the time that the backend need for processing.
There are multiple things which needs to be considered while calculating Round Trip Time(RTT) like
Application architecture
Cold start duration
Server response time
Apart from the above factors there will be multiple other factors as well, but for now let's concentrate on these factors alone because these are directly related to the issue which you are asking.
As you mentioned, if we monitor the load balancer IP alone it’s not going to help us find the RTT properly. To calculate the RTT properly we need to add a few functions in our code which will assist in collecting the response timestamps using which we can calculate correct RTT. If you are already having your response logs generated for your cloud run deployment you can compare the difference between the timestamps for calculating the exact RTT taken.
If you are calculating RTT for fixed locations you can use the GCP’s Network Intelligence Centre dashboard for collecting insights about various geo locations as mentioned in this blog written by Gauravmadan.
In the question it is mentioned that you are using classic HTTPS load balancer instead of using Global HTTPS load balancer. Classic load balancer doesn’t support geo load balancing, as a result there might be a chance that the latency will be much higher. If you are planning to have an application with global presence it is suggested to use Global HTTPS load balancers over the classic HTTPS load balancer.