How to prevent sidekiq taking 18 seconds on average to pick up new job?

Question

I have a Rails 7 API running on Heroku with a web and worker dyno. The frontend calls a conversation create controller action, which triggers a job for the worker. Then immediately after, the frontned calls a conversation_entry create action, which triggers a job for the worker again.

The first job starts immediately as the action is called, executes and completes it. The next job takes ~20 seconds to start executing despite the controller action having completed a long time ago.

How do I shorten the amount of time it takes for Sidekiq to pick up a new job and not just sit idle?

I tried scaling up to 2 workers on Heroku, which didn't help.

Relevant code below:

sidekiq.rb

require 'sidekiq'

Sidekiq.configure_client do |config|
  config.redis = { :size => 1 }
end

Sidekiq.configure_server do |config|
  config.redis = { :size => 12 }
  config.average_scheduled_poll_interval = 5
  config.concurrency = 5
end

sidekiq.yml

:concurrency: 5
:pidfile: ./tmp/pids/sidekiq.pid
:queues:
  - [urgent, 8]
  - [high, 5]
  - [default, 3]
:production:
  url: <%= ENV['REDIS_URL'] %>
  pool_size: 12
:development:
  :concurrency: 10
  :url: redis://localhost:3001/0

Answer 1

Ok, issue was that the second job was reliant on some information being pushed to the db from first job. First job was slower than second job, so second job failed and only on retry (15 sec after) it suceeded.