Memory usage increase with Ruby 2.1 versus Ruby 2.0 or 1.9

Question

I recently deployed an upgrade of Ruby from 2.0 to 2.1.5 to my Heroku web application, and I am consistently hitting memory quota errors now, whereas with 2.0 and 1.9 this never happened. There is a limit of 512MB for a normal Heroku Dyno, and I am running 2 processes with Unicorn, along with one thread with Sidekiq across two dynos.

After reading Phusion Passenger memory consumption increase from 1.9.3 (system) to 2.1.2 (RVM) on Ubuntu, I tried setting the environment variable RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR to 0.9 to disable generational garbage collector, and it did improve memory consumption somewhat, but still left me nowhere near what 2.0 or 1.9 consumed. As memory right now is my primary concern, I would like to see if I can fix this problem with ruby 2.1.x instead of reverting to 2.0.

Here are some graphs demonstrating the problem:

At just before 2pm, I downgraded to Ruby 2.0.0-p598 to Ruby 2.1.5 and the memory issues were resolved and remained under the limit.

Then later, I tried upgrading to Ruby 2.1.5 but setting the environment variable RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR to 0.9. As you can see, the memory usage is slightly better than the original 2.1.5 graph, but still breaches the memory quota.

Answer 1

This is a known issue with Ruby (versions 2.1.x) and its garbage collection. After reading couple of forums/blog posts there seems to be no real solution but to do one of the following:

• downgrade to ruby 2.0 and wait for ruby 2.2 to be released
• use unicorn worker killer to restart unicorn workers once they reach certain amount of memory, preventing Heroku R14 errors
• some suggest tweaking your GC variables (see here and here)

Here's a link to relevant discussion on ruby-lang.

Some people also noticed that one source of the problems could be NewRelic's gem so you might want to update/remove it and see if it helps.

Credits for some of the links and info to guys from upcase.com forum

Answer 2

So the answer is clearly upgrade to Ruby version 2.2.

I am impressed to see that memory usage has actually gone down following the upgrade to 2.2, whereas upgrading to 2.1 was a huge increase in memory consumption. See the graph below.

On the far left memory consumption on 2.0 sits at around 386mb when under load, whereas with 2.2 it now sits at around 365mb.