Get UDP packets from the same client delivered to the same server process, when using SO_REUSEPORT?

Question

I'm writing a multi-process UDP server, which uses SO_REUSEPORT to let multiple worker processes listen on the same port.

Is there a way of telling the kernel that I would like consistent delivery of packets to processes, based on the packets' source address?

(The server needs to hold a little state, per source address. It would be easier to have state local to the processes, rather than for state to have to be shared between processes.)

Answer 1

You might be able to achieve a similar result using a BPF with the SO_ATTACH_REUSEPORT_CBPF or SO_ATTACH_REUSEPORT_EBPF socket options to assign each client to a specific socket index.

Answer 2

You should take a look and configure RPS: Receive Packet Steering:

The first step in determining the target CPU for RPS is to calculate a flow hash over the packet’s addresses or ports (2-tuple or 4-tuple hash depending on the protocol). This serves as a consistent hash of the associated flow of the packet. The hash is either provided by hardware or will be computed in the stack.

...

Each receive hardware queue has an associated list of CPUs to which RPS may enqueue packets for processing. For each received packet, an index into the list is computed from the flow hash modulo the size of the list. The indexed CPU is the target for processing the packet, and the packet is queued to the tail of that CPU’s backlog queue.

...

RPS scales kernel receive processing across CPUs without introducing reordering. The trade-off to sending all packets from the same flow to the same CPU is CPU load imbalance if flows vary in packet rate.

Another option is Intel Ethernet Flow Director:

Intel Ethernet FD supports advanced filters that direct received packets to different queues, and enables tight control on flow in the platform. It matches flows and CPU cores where the processing application is running for flow affinity, and supports multiple parameters for flexible flow classification and load balancing. When operating in Application Targeting Routing (ATR) mode, Intel Ethernet FD is essentially the hardware offloaded version of Receive Flow Steering available on Linux* systems, and when running in this mode, Receive Packet Steering and Receive Flow Steering are disabled.