Dynamically create public IPs or subdomains for EKS pods

Question

Complex AWS EKS / ENI / Route53 issue has us stumped. Need an expert.

Context:

We are working on dynamic game servers for a social platform (https://myxr.social) that transport game and video data using WebRTC / UDP SCTP/SRTP via https://MediaSoup.org

Each game server will have about 50 clients Each client requires 2-4 UDP ports

Our working devops strategy https://github.com/xr3ngine/xr3ngine/tree/dev/packages/ops

We are provisioning these game servers using Kubernetes and https://agones.dev

Mediasoup requires each server connection to a client be assigned individual ports. Each client will need two ports, one for sending data and one for receiving data; with a target maximum of about 50 users per server, this requires 100 ports per server be publicly accessible.

We need some way to route this UDP traffic to the corresponding gameserver. Ingresses appear to primarily handle HTTP(S) traffic, and configuring our NGINX ingress controller to handle UDP traffic assumes that we know our gameserver Services ahead of time, which we do not since the gameservers are spun up and down as they are needed.

Questions:

We see two possible ways to solve this problem.

Path 1

Assign each game server in the node group public IPs and then allocate ports for each client. Either IP v4 or v6. This would require SSL termination for IP ports in AWS. Can we use ENI and EKS to dynamically create and provision IP ports for each gameserver w/ SSL? Essentially expose these pods to the internet via a public subnet with them each having their own IP address or subdomain. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html We have been referencing this documentation trying to figure out if this is possible.

Path 2

Create a subdomain (eg gameserver01.gs.xrengine.io, etc) dynamically for each gameserver w/ dynamic port allocation for each client (eg client 1 [30000-30004], etc). This seems to be limited by the ports accessible in the EKS fleet.

Are either of these approaches possible? Is one better? Can you give us some detail about how we should go about implementation?

Answer 1

The native way for receiving UDP traffic on Amazon EKS is by using a Kubernetes Service of type Loadbalancer with an extra annotation to get the NLB.

Example

apiVersion: v1
kind: Service
metadata:
  name: my-game-app-service
  annotation:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
spec:
  selector:
    app: my-game-app
  ports:
    - name: outgoing-port   # choose your name
      protocol: UDP
      port: 9000            # choose your port
    - name: incoming-port   # choose your name
      protocol: UDP
      port: 9001            # choose your port
  type: LoadBalancer