We're running mod_perl on Apache 2 and get seemingly random header related errors that we just can't figure out. Due to the nature of the site we get hit by a ton of bots, so I'm thinking these are caused by bad or malformed requests from bots, but I'd like to figure it out for sure one way or another so I know where to go from here. Here's an example of the 2 most common errors we see in the logs:
[Thu Nov 13 21:40:48 2014] [warn] /whatever did not send an HTTP header
[Thu Nov 13 21:40:48 2014] [error] [client x] malformed header from script. Bad header=\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z: index.cgi
[Fri Nov 14 00:04:17 2014] [warn] /whatever did not send an HTTP header
[Fri Nov 14 00:04:17 2014] [error] [client x] Premature end of script headers: index.cgi
We get 100s of 1,000s of requests to these same URLs daily, and they work fine 99.999% of the time. I don't believe it's our scripts - we always output correct headers. No real users have ever complained about any errors on our site, etc. so I'm hoping this is just caused by some bad requests from bots.
And if so, what if anything can we do to make these stop? It's a real pain because these errors trip our monitoring systems and my techie gets about 20-30 fake error alerts every day.
Turns out it was a problem with Safari browsers and mod_deflate compression.
The simple solution:
BrowserMatch Safari gzip-only-text/html