Error in exposing multiple ports with ALB Ingress on EKS

Question

I have a Triton server on EKS listening on 3 ports, 8000 is for http requests, 8001 is for gRPC and 8002 is for prometheus metrics. So, I have created a Triton deployment on EKS which is exposed through NodePort service of EKS. I am also using ALB ingress which is creating an application load balancer to balance the load of Triton servers on these ports.

But the traffic is not flowing correctly as it is showing same output for all the 3 ports but it should be different. So, now do I have to create 3 Application Load Balancers for 3 ports or is it possible to manage all ports with a single Application Load Balancer?

Yaml file for ALB Ingress looks like:-

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: triton
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8000}, {"HTTP":8001}, {"HTTP":8002}]'
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: triton
              port:
                number: 8000
    - http:
       paths:
       - path: /v2
         pathType: Prefix
         backend:
            service:
              name: triton
              port:
                number: 8001
    - http:
        paths:
        - path: /metrics
          pathType: Prefix
          backend:
            service:
              name: triton
              port:
                number: 8002

Answer 1

Based on the Load Balancer controller's documentation, the listen ports specified on the annotation alb.ingress.kubernetes.io/listen-ports is merged across all Ingresses, so the rules to all of the ports are going to be applied on all the listeners.

This is going to require a deployment of multiple Ingresses in the Kubernetes cluster. By default, if you deploy multiple Ingress Controllers in the EKS, then whenever you create an Ingress, a race condition will occur between all those controllers in which each controller tries to update Ingress status fields. Hence Kubernetes provides the capability where different Ingresses can be implemented by different Ingress controllers. This is achieved by using IngressClass. Than we can share the same ALB by specifying the alb.ingress.kubernetes.io/group.name annotation.

The code should be something like this:

---
apiVersion: networking.k8s.io/v1
kind: IngressClass 
metadata:
  name: alb
spec:
  controller: ingress.k8s.aws/alb
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: triton
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8000}]'
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/group.name: triton-group
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: triton
              port:
                number: 8000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: triton
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8001}]'
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/group.name: triton-group
spec:
  ingressClassName: alb
  rules:
    - http:
       paths:
       - path: /v2
         pathType: Prefix
         backend:
            service:
              name: triton
              port:
                number: 8001
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: triton
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":8002}]'
    alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    alb.ingress.kubernetes.io/group.name: triton-group
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /metrics
          pathType: Prefix
          backend:
            service:
              name: triton
              port:
                number: 8002