dial tcp: lookup kfsoftware.github.io on 10.43.0.10:53 server misbehaving

Question

Error: looks like "https://kfsoftware.github.io/hlf-helm-charts" is not a valid chart repository or cannot be reached: GET "https://kfsoftware.github.io/hlf-helm-charts/index.yaml": dial tcp: lookup kfsoftware.github.io on 10.43.0.10:53 server misbehaving.

Getting this error from the kubernetes pod while trying to add helm repo within the pod. This error is coming from a private network. I checked the internet connection is there on the machine.

I've created k8s cluster using rke rancher.

I tried doing exec into the pod and running help cmds manually, that is also not working even ping is also not responding. But the VM (where k8s is installed) i'm able to run the cmds and do the ping. I think there is some error with the kuberntes network config.

Any Help is highly appreciated.

Answer 1

even ping is also not responding

Maybe the DNS resolution inside the pod is not working or is disabled/custom? Port 53 is usually used for this so this might be your case as you pasted 10.43.0.10:53

The IP for kfsoftware.github.io is as follows:

$ nslookup kfsoftware.github.io
Server:     1.1.1.1
Address:    1.1.1.1#53

Non-authoritative answer:
Name:   kfsoftware.github.io
Address: 185.199.111.153
Name:   kfsoftware.github.io
Address: 185.199.108.153
Name:   kfsoftware.github.io
Address: 185.199.110.153
Name:   kfsoftware.github.io
Address: 185.199.109.153

Check what DNS policy is set for that pod:

https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-s-dns-policy

you might want to try this in pod manifest :

apiVersion: v1
kind: Pod
metadata:
  name: bar
  namespace: default
spec:
  containers:
  - image: foo
    name: bar
  dnsPolicy: "None"
  dnsConfig:
    nameservers:
      - 1.1.1.1 #cloudflare