Rails Puma and 3000 concurrent users - Slow as molasses

Question

We are using Neustar to test how our servers can handle loads. Our goal is to hit around 3000 concurrent users.

We have a very simple php application (no framework, served up by nginx, using mysql) that handled this without an issue. Response times were near flat, regardless of users at just above 300ms.

Same test with Rails, puma, and nginx, and response times are ABYSMAL. At around 500 users response times begins to climb to 1 second, then beings a relatively linear climb to around 4 seconds at around 1000, and 6 seconds around 2000.

When I check the logs, I get reports that indicate that we're responding in significantly shorter periods of time:

Completed 200 OK in 8.8ms (Views: 0.2ms | ActiveRecord: 4.9ms)
Started GET "/sessions/319/questions/poll.json" for IPADDRESS at 2013-12-30 22:17:28 +0000
Processing by QuestionsController#poll as JSON
  Parameters: {"session_id"=>"319"}
Started GET "/sessions/319/questions/poll.json" for IPADDRESS at 2013-12-30 22:17:28 +0000
Processing by QuestionsController#poll as JSON
  Parameters: {"session_id"=>"319"}
Completed 200 OK in 436.0ms (Views: 208.8ms | ActiveRecord: 212.6ms)
Processing by QuestionsController#poll as JSON
  Parameters: {"session_id"=>"319"}
Started GET "/sessions/319/questions/get_current.json" for IPADDRESS at 2013-12-30 22:17:28 +0000
Started GET "/sessions/319/questions/poll.json" for IPADDRESS at 2013-12-30 22:17:28 +0000
Processing by QuestionsController#get_current as JSON
Completed 200 OK in 125.2ms (Views: 48.2ms | ActiveRecord: 63.4ms) 

We're a bit new to this whole thing, and I can't help but feel we're missing out on something very fundamental, because this seems completely out of whack.

If I can provide any more helpful information, please let me know.

Any help, suggestions, blatently obvious pieces of setup that we've missed, would be most appreciated.

Thank you.

Answer 1

This is not a solution for getting PUMA to work for our requirements, but I figured that I'd give some closure to future readers.

We ended up using PHP, php-fpm, and nginx.

The same server that could barely handle around 1000 users with rails and PUMA was easily able to handle 3000+.

NOTE: This is not to say that PUMA can't scale, merely that for our situation (A single powerful box running both MYSQL AND the application), php performed adequately.

I'm not sure why this is yet, but I've heard that this is because rails tends to want to scale ACROSS servers (eg. multiple servers), while PHP does better at scaling vertically.

If anyone has any corrections, additional information, or comments, I'd love to hear them.