I have Rack::Cache set up as the very top (last) middleware in my stack. My app is hosted on heroku cedar, so Rack::Cache is responsible for static assets as well.
It's backed by the heroku memcached product, and configured like this:
config.action_dispatch.rack_cache = {
:metastore => Dalli::Client.new,
:entitystore => 'file:tmp/cache/rack/body',
:allow_reload => false
}
config.static_cache_control = "public, max-age=2592000"
It's running on thin.
I'm testing performance on an image file, using ab.
ab -n100 -c10 https://example.com/foo.jpg
Looking in my logs, all the requests are a cache hit ("fresh"). But the performance is poor, only around 6 requests/sec. I know that Rack::Cache isn't going to perform as well as a dedicated http proxy, but I certainly expected higher throughput than this.
update I don't how the heck I didn't think of this before, but my benchmark is indeed maxing out my internet connection. If I do the same test with robots.txt instead of a jpg, I get 20 requests/sec (and still max out the downstream).
I noticed the URL in your post is HTTPS. I had an issue on the cedar stack with hostname-based SSL where certain requests (for both static assets and otherwise) would experience high latency even though the logs said the queue time was 0ms and the service time was 5-10ms. Here's Heroku's support response (what he doesn't mention is that varnish doesn't cache resources coming from cedar apps, regardless, despite it being present in the stack when running hostname based SSL):
Granted, the latencies I were seeing sometimes got up into the 30-60 SECOND range, which is obviously quite a bit worse than you're seeing. When I run the same apache bench commands against static assets on some of my (HTTPS) cedar apps, I don't do much better (10-15 req/s). When I increase concurrency to 50 or 100, RPS goes up to as much as 30. On Varnish fronted bamboo HTTPS apps, however, I can get it up over 50 req/s for a static asset. If my computer could handle higher concurrency, I suspect it would go even higher. And my internet connection is high latency since it's on a cellular network.