High ping latency between two machines

Question

I am trying to ping a machine over tailscale and seeing very high latency numbers (> 3000 ms). The machines are very closely geo located. One of them is behind a Comcast router, and another is behind AT&T Fiber. How should I go about debugging this?

Answer 1

If your traffic is routing through a DERP as explained in Tailscale's troubleshooting guide, you may need to open one or more ports in order to connect your machines directly.

Use the Tailscale CLI to run the tailscale status command. If you see output in the form of relay "code", then your traffic is being routed via a relay server that has “code” as its location.

If there is no relay "code" line in the tailscale status output, then your traffic is not being routed through DERP.

Also, the tailscale ping command will indicate whether a successful ping was by direct path or via DERP.

Example:

tailscale ping node2

pong from node2 (100.99.98.96) via DERP(sea) in 242ms
pong from node2 (100.99.98.96) via 1.2.3.4:1234 in 127ms

Answer 2

Do a traceroute to the machine. Tailscale will very often fallback to a cloud relay which can add a lot of latency to the connection even when they are physically close.