add latency with tc without restricting bandwidth

Question

I'm trying to use tc to add latency to responses from a webserver in order to simulate a WAN. I found a few related posts and tried out the command:

tc qdisc add dev eth0 root netem delay 100ms

I am using a 10G NIC to make a high volume of requests equalling about 3Gbps. After using tc to add latency, I see a massive drop off in throughput, and the latency of responses gets closer to about 3 seconds.

Am I missing something in the above command, it it limiting the rate / throughput in addition to adding latency?

N.B tc qdisc returns the following:

qdisc netem 8005: dev eth0 root refcnt 72 limit 1000 delay 100.0ms  10.0ms rate 10000Mbit

Answer 1

you can now set socket option with maximum pacing rate, TCP is going to take care to rate limit, alternatively you can use fq qdisc, which supports packet timestamp in form of earliest departure time (EDT), you can play with timestamp using a clsact bpf filter.

some reference- https://legacy.netdevconf.info/0x14/pub/slides/55/slides.pdf

Answer 2

Firstly, I think tc cannot process packets at such high data rates. I experienced drop in throughput as well when I was playing with it few years ago. I used both 10GbE and 40GbE.

Unfortunately, I have no access to such hardware now.

I would suggest you to check the buffer sizes as you are emulating a delay of 100ms. packets are getting dropped somewhere and affecting your throughput. The increased latency can be because of the packet making it to the destination after being dropped many times (small buffer size) or being queued for a ver long time (very large buffer size)