Increase ephemeral ports by adding extra ethernet interface

Question

Is there a way to double the number of ephemeral ports and work around the 16-bit limit? I have tried creating virtual ethernet interfaces over eth0, and hope that would lift the limit. Although the application is utilizing the new virtual IPs in outbound traffic, it seems still hitting the same ephemeral-port limit. I suppose the virtual ports have 1-to-1 mapping to the ports on the physical interface.

ifconfig eth0:1 10.10.10.210 netmask 255.255.255.192 ifconfig eht0:2 10.10.10.211 netmask 255.255.255.192

Could someone please advise how I could double the total number of ephemeral ports in Linux, without adding an extra NIC?

(FYI, I have tried increasing ulimit/max open file, changing the port range, enabling tcp recycle/timestamps, reducing tcp fin timeout... I suppose we simply need more than 65k ports for this proxy machine.)

Answer 1

It's a limitation of the network protocols. Both TCP & UDP, for instance, have 16 bit source and destination ports. Even if you could increase the number of ports no one could address them.

Answer 2

If you create virtual interfaces over eth0, then you should be able to assign different IP addresses to those interfaces. With that, you can use the same ephemeral port numbers (they are allocated in the kernel, so you dont really have much control) for multiple sockets each bound to different addresses -- you will probably need to set SO_REUSEADDR option. The reason this will work is because for incoming packets (UDP/TCP), the flow is identified by looking at both local source IP and the port number.

And as @Duck mentioned, since TCP/UDP headers allocate only 16 bits for port numbers, there is not much point in increasing the ephemeral range in the local stack.

Answer 3

It turns out that you cannot use 0 for binding ephemeral port if you want to exceed the 65535 limit. Instead, you need to use an explicit port number.

And also turning on tcp_tw_reuse might be helpful: http://krenel.org/tcp-time_wait-and-ephemeral-ports-bad-friends.html

Answer 4

It seems there's a way, but it's not for free. It's called "bind before connect". See this short but dense article, which sums it up very nicely.

Having multiple virtual IPs is just a start. Quoting the linked article:

On Linux the ephemeral port range is a global resource, it's not a specific setting local to an IP address.

So that's bad and you have to improve your starting position with few right settings (where most of them you already found) and get around the global limit with a clever socket allocation technique. The result is that you'll control all the outgoing IPs manually. This also seems not to cope well with other apps on the system using the traditional "connect" way.