How to connect front to back in k8s cluster internal (connection refused)

Question

Error while trying to connect React frontend web to nodejs express api server into kubernetes cluster.

Can navigate in browser to http:localhost:3000 and web site is ok.

But can't navigate to http:localhost:3008 as expected (should not be exposed)

My goal is to pass REACT_APP_API_URL environment variable to frontend in order to set axios baseURL and be able to establish communication between front and it's api server.

deploy-front.yml

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: gbpd-front
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: frontend
      track: stable
  replicas: 2
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: frontend
        track: stable
    spec:
      containers:
        - name: react
          image: binomio/gbpd-front:k8s-3
          ports:
            - containerPort: 3000
          resources:
            limits:
              memory: "150Mi"
            requests:
              memory: "100Mi"
          imagePullPolicy: Always

service-front.yaml

apiVersion: v1
kind: Service
metadata:
  name: gbpd-front
spec:
  selector:
    app: gbpd-api
    tier: frontend
  ports:
  - protocol: "TCP"
    port: 3000
    targetPort: 3000
  type: LoadBalancer

Deploy-back.yaml

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: gbpd-api
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: backend
      track: stable
  replicas: 3 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: backend
        track: stable
    spec:
      containers:
        - name: gbpd-api
          image: binomio/gbpd-back:dev
          ports:
            - name: http
              containerPort: 3008

service-back.yaml

apiVersion: v1
kind: Service
metadata:
  name: gbpd-api
spec:
  selector:
    app: gbpd-api
    tier: backend
  ports:
  - protocol: "TCP"
    port: 3008
    targetPort: http

I tried many combinations, also tried adding "LoadBalancer" to backservice but nothing...

I can connect perfecto to localhost:3000 and use frontend but frontend can't connect to backend service.

Question 1: What's is the ip/name to use in order to pass REACT_APP_API_URL to fronten correctly? Question 2: Why is curl localhost:3008 not answering?

After 2 days trying almost everything in k8s official docs... can't figure out what's happening here, so any help will be much appreciated.

kubectl describe svc gbpd-api Response:

kubectl describe svc gbpd-api
Name:                     gbpd-api
Namespace:                default
Labels:                   <none>
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"gbpd-api","namespace":"default"},"spec":{"ports":[{"port":3008,"p...
Selector:                 app=gbpd-api,tier=backend
Type:                     LoadBalancer
IP:                       10.107.145.227
LoadBalancer Ingress:     localhost
Port:                     <unset>  3008/TCP
TargetPort:               http/TCP
NodePort:                 <unset>  31464/TCP
Endpoints:                10.1.1.48:3008,10.1.1.49:3008,10.1.1.50:3008
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

Answer 1

I tested your environment, and it worked when using a Nginx image, let's review the environment:

• The front-deployment is correctly described.
• The front-service exposes it as loadbalancer, meaning your frontend is accessible from outside, perfect.
• The back deployment is also correctly described.
• The backend-service stays with as ClusterIP in order to be only accessible from inside the cluster, great.

Below I'll demonstrate the communication between front-end and back end.

• I'm using the same yamls you provided, just changed the image to Nginx for example purposes, and since it's a http server I'm changing containerport to 80.

Question 1: What's is the ip/name to use in order to pass REACT_APP_API_URL to fronten correctly?

• I added the ENV variable to the front deploy as requested, and I'll use it to demonstrate also. You must use the service name to curl, I used the short version because we are working in the same namespace. you can also use the full name: http://gbpd-api.default.svc.cluster.local:3008

---

Reproduction:

• Create the yamls and applied them:

$ cat deploy-front.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gbpd-front
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: frontend
      track: stable
  replicas: 2
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: frontend
        track: stable
    spec:
      containers:
        - name: react
          image: nginx
          env:
            - name: REACT_APP_API_URL
              value: http://gbpd-api:3008
          ports:
            - containerPort: 80
          resources:
            limits:
              memory: "150Mi"
            requests:
              memory: "100Mi"
          imagePullPolicy: Always

$ cat service-front.yaml 
cat: cat: No such file or directory
apiVersion: v1
kind: Service
metadata:
  name: gbpd-front
spec:
  selector:
    app: gbpd-api
    tier: frontend
  ports:
  - protocol: "TCP"
    port: 3000
    targetPort: 80
  type: LoadBalancer

$ cat deploy-back.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gbpd-api
spec:
  selector:
    matchLabels:
      app: gbpd-api
      tier: backend
      track: stable
  replicas: 3
  template:
    metadata:
      labels:
        app: gbpd-api
        tier: backend
        track: stable
    spec:
      containers:
        - name: gbpd-api
          image: nginx
          ports:
            - name: http
              containerPort: 80

$ cat service-back.yaml 
apiVersion: v1
kind: Service
metadata:
  name: gbpd-api
spec:
  selector:
    app: gbpd-api
    tier: backend
  ports:
  - protocol: "TCP"
    port: 3008
    targetPort: http

$ kubectl apply -f deploy-front.yaml 
deployment.apps/gbpd-front created
$ kubectl apply -f service-front.yaml 
service/gbpd-front created
$ kubectl apply -f deploy-back.yaml 
deployment.apps/gbpd-api created
$ kubectl apply -f service-back.yaml 
service/gbpd-api created

• Remember, in Kubernetes the communication is designed to be made between services, because the pods are always recreated when there is a change in the deployment or when the pod fail.

$ kubectl get all
NAME                              READY   STATUS    RESTARTS   AGE
pod/gbpd-api-dc5b4b74b-kktb9      1/1     Running   0          41m
pod/gbpd-api-dc5b4b74b-mzpbg      1/1     Running   0          41m
pod/gbpd-api-dc5b4b74b-t6qxh      1/1     Running   0          41m
pod/gbpd-front-66b48f8b7c-4zstv   1/1     Running   0          30m
pod/gbpd-front-66b48f8b7c-h58ds   1/1     Running   0          31m

NAME                 TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)          AGE
service/gbpd-api     ClusterIP      10.0.10.166   <none>         3008/TCP         40m
service/gbpd-front   LoadBalancer   10.0.11.78    35.223.4.218   3000:32411/TCP   42m

• The pods are the workers, and since they are replaceable by nature, we will connect to a frontend pod to simulate his behaviour and try to connect to the backend service (which is the network layer that will direct the traffic to one of the backend pods).
• The nginx image does not come with curl preinstalled, so I will have to install it for demonstration purposes:

$ kubectl exec -it pod/gbpd-front-66b48f8b7c-4zstv -- /bin/bash
root@gbpd-front-66b48f8b7c-4zstv:/# apt update && apt install curl -y
done.

root@gbpd-front-66b48f8b7c-4zstv:/# curl gbpd-api:3008
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...

• Now let's try using the environment variable that was defined:

root@gbpd-front-66b48f8b7c-4zstv:/# printenv | grep REACT
REACT_APP_API_URL=http://gbpd-api:3008
root@gbpd-front-66b48f8b7c-4zstv:/# curl $REACT_APP_API_URL
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...

---

Considerations:

Question 2: Why is curl localhost:3008 not answering?

• Since all yamls are correctly described you must check if image: binomio/gbpd-back:dev is correctly serving on port 3008 as intended.
• Since it's not a public image, I can't test it, so I'll give you troubleshooting steps:
  • just like we logged inside the front-end pod you will have to log into this backend-pod and test curl localhost:3008.
  • If it's based on a linux distro with apt-get, you can run the commands just like I did on my demo:
  • get the pod name from backend deploy (example: gbpd-api-6676c7695c-6bs5n)
  • run kubectl exec -it pod/<POD_NAME> -- /bin/bash
  • then run apt update && apt install curl -y
  • and test curl localhost:3008
  • if no answer run `apt update && apt install net-tools
  • and test netstat -nlpt, it will have to show you the output of the services running and the respective port, example:

root@gbpd-api-585df9cb4d-xr6nk:/# netstat -nlpt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1/nginx: master pro 

• If the pod does not return nothing even on this approach, you will have to check the code in the image.

Let me know if you need help after that!