I've seen a very strange phenomenom I don't understand. I'm trying to debug a slow download speed from one of my server, and I use the following command from an external machine:
curl -w "@aa" -o /dev/null 'http://myserver/bigfile'
where the file aa contains reporting format as shown below:
DNS lookup : 0.000
Connect to server (TCP) : 0.004
Connect to server (HTTP/S) : 0.000
Time from start until transfer began: 0.004
Time for redirection (if any) : 0.000
Total time before transfer started : 0.006
Total time : 147.207
Size of download (bytes) : 118263829
Average d/l speed (bytes/s) : 803383.000
You can see the download rate is "slow:" 800 kB/s
But when I just add the --trace option:
curl --trace /dev/null -w "@aa" -o /dev/null 'http://myserver/bigfile'
I've got:
DNS lookup : 0.000
Connect to server (TCP) : 0.007
Connect to server (HTTP/S) : 0.000
Time from start until transfer began: 0.007
Time for redirection (if any) : 0.000
Total time before transfer started : 0.008
Total time : 25.430
Size of download (bytes) : 118263829
Average d/l speed (bytes/s) : 4650587.000
Which is way faster ! (4.6MB/s)
How is it possible that just by adding the --trace option, download speed increases a lot ??
(additional note: it's even faster with the --trace-ascii option: up to 10MB/s !) I guess this problem should be related to the web server configuration, but I'm not sure and I'd like to know if someone has already observed a similar behavior, or if you have an idea about which part of the apache2 configuration may explain this... Thanks !
New information
I've compared the tcpdump outputs in both cases, and I've observed that, without the --trace-ascii option, there are many more repeated ACK packets, such as:
Flags [.], ack 444361, win 1392, length 0
Flags [.], ack 444361, win 1392, length 0
Flags [.], ack 444361, win 1392, length 0
and also many more ACK-SACK, such as:
Flags [.], ack 445649, win 1414, options [nop,nop,TS val 938602294 ecr 2998737861,nop,nop,sack 3 {108830077:108831365}{108818485:108819773}{108821061:108824925}], length 0
There is nothing like repeated ACK nor ACK-SACK packets with the --trace-ascii option. So I guess the problem is related to lost packets in the slowest case. But both machines and networks are exactly similar, and I've repeated both experiments many times, so the only difference comes from the --trace-ascii option.