I'm facing k8s (server version v1.19.9) request slowness, so I checked the log of the kube-apiserver pod and found the following related log
I1127 00:52:56.466386 1 httplog.go:89] "HTTP" verb="GET" URI="/api/v1/namespaces/xxx/resourcequotas" latency="1.497384ms" userAgent="xxx/v0.0.0 (linux/amd64) kubernetes/$Format" srcIP="xxx" resp=200
As the log indicates, the latency is quite small (latency="1.497384ms"), makes wondering if the latency in the log contains the time of Request Handling Queue. So I've checked the source code https://github.com/kubernetes/kubernetes/blob/v1.19.9/staging/src/k8s.io/apiserver/pkg/server/httplog/httplog.go#L173 , but still can't understand if it includes Request Handling Queue time, anyone has insights it?
Btw, the kube-apiserver is lauched with default max-requests-inflight (400), max-mutating-requests-inflight (200) and enable-priority-and-fairness (true). And according to Datadog, the kube-apiserver:current_inflight_requests KPI were always less than 5.
Also, it is not slow if I execute kubectl command from my laptop, it is only slow when the request is from a Golang microservice with kubernetes.Clientset