How to measure network pressure?

Question

Let's say I'm building my own download accelerator.

Let's simplify it to the point where:

• my code runs at 3rd party whose network parameters I cannot control
• an item is downloaded from a single IP
• number of parallel range transfers is adjustable
• there will be many transfers to learn ideal parameters
• client runs Linux
• server is outside my control
• path is over WAN and download is over HTTPS
• downloaded segments are large

How do I measure if enough connections are used to saturate the path between client and server?

What bits from getsockopt(..., TCP_INFO) are actually useful?

How fast can I adjust to varying network conditions?

It's possible to measure CPU and memory pressure on a client system, how about network pressure?

Answer 1

The answer is very simple: One TCP connection is sufficient to saturate the path between client and server.

At least, that is true with standard TCP/IP protocols and standard network devices between the client and server. If you have some devices in the path that are doing some sort of customized processing that is "throttling" individual TCP connections then maybe the answer is different. But in that case it's impossible to answer unless you specify exactly what that custom processing is. As others have commented, in most cases network traffic shaping or limiting is probably being done based on IP addresses rather than on the TCP headers, so creating additional TCP connections isn't going to help. If your case is different, that might be interesting to hear about - please explain further.

In my opinion (just an opinion, you may disagree, of course) the real answer to a question about building a custom "download accelerator" is: Don't do that. TCP works very well just as it is. Apart from a few special cases (e.g. SCPS-TP) you are unlikely to improve TCP performance by playing tricks with multiple connections or messing with socket options. And if you do manage to improve performance, you are probably doing it at the expense of degrading overall network performance for other users. That's ill-advised.