Rails cached value lost/nil despite expires_in 24.hours

Question

I am using ruby 2.3.3 and Rails 4.2.8 with Puma (1 worker, 5 threads) and on my admin (i.e. not critical) page I want to show some stats (integer values) from my database. Some requests take quite a long time to perform so I decided to cache these values and use a rake task to re-write them every day.

Admin#index controller

require 'timeout'
begin
  timeout(8) do
    @products_a = Rails.cache.fetch('products_a', :expires_in => 24.hours) { Product.where(heavy_condition_a).size }

    @products_b = Rails.cache.fetch('products_b', :expires_in => 24.hours) { Product.where(heavy_condition_b).size }
    @products_c = Rails.cache.fetch('products_c', :expires_in => 24.hours) { Product.where(heavy_condition_c).size }
    @products_d = Rails.cache.fetch('products_d', :expires_in => 24.hours) { Product.where(heavy_condition_d).size }              
  end
rescue Timeout::Error
  @products_a = 999
  @products_b = 999
  @products_c = 999
  @products_d = 999   

end

Admin#index view

    <li>Products A: <%= @products_a %></li>
    <li>Products B: <%= @products_b %></li>
    <li>Products C: <%= @products_c %></li>
    <li>Products D: <%= @products_d %></li>

Rake task

task :set_admin_index_stats => :environment do
    Rails.cache.write('products_a', Product.where(heavy_condition_a).size, :expires_in => 24.hours)
    Rails.cache.write('products_b', Product.where(heavy_condition_b).size, :expires_in => 24.hours)
    Rails.cache.write('products_c', Product.where(heavy_condition_c).size, :expires_in => 24.hours)
    Rails.cache.write('products_d', Product.where(heavy_condition_d).size, :expires_in => 24.hours)                     
end

I am using this in production and use Memcachier (on Heroku) as a cache store. I also use it for page caching on the website and it works fine there. I have:

production.rb

config.cache_store = :dalli_store

The problem I am experiencing is that the cached values disappear almost instantly, and quite intermittently, from the cache. In the console I have tried:

1. I Rails.cache.write one value (e.g. product_a) and check it a minute later, it is still there. Although crude, I can see the "Set cmds" increments by one in Memcachier admin tool.
2. However, when I add the next value (e.g. product_b) the first one disappears (becomes nil)! Sometimes if I add all 4 values, 2 seems to stick. These are not always the same values. It is like whack-a-mole!
3. If I run the rake to write the values and then try to read the values typically only two values are left, whereas the others are lost.

I have seen a similar question related to this where the reason explained was the use of a multithread server. The cached value was saved in one thread and could not be reached in another, the solution was to use a memcache, which I do.

It is not only the console. If I just reload admin#index view to store the values or run the rake task, I experience the same problem. The values do not stick.

My suspicion is that I am either not using the Rails.cache-commands properly or that these commands do not in fact use Memcachier. I have not been able to determine whether my values are in fact stored in Memcachier but when I use my first command in the console I do get the following:

Rails.cache.read('products_a')
Dalli::Server#connect mc1.dev.eu.ec2.memcachier.com:11211
Dalli/SASL authenticating as abc123
Dalli/SASL authenticating as abc123
Dalli/SASL: abc123
Dalli/SASL: abc123
=> 0

but I do not get that for subsequent writes (which I assume is a matter of readability in the console and not a proof of Memcachier not being used.

What am I missing here? Why won't the values stick in my cache?

Answer 1

Heroku DevCenter states a little different cache config and gives some advice about threaded Rails app servers like Puma using connection_pool gem:

config.cache_store = :mem_cache_store,
  (ENV["MEMCACHIER_SERVERS"] || "").split(","),
  { :username => ENV["MEMCACHIER_USERNAME"],
    :password => ENV["MEMCACHIER_PASSWORD"],
    :failover => true,
    :socket_timeout => 1.5,
    :socket_failure_delay => 0.2,
    :down_retry_delay => 60,
    :pool_size => 5
  }