How to assure the same job doesn't run twice at the same time?

Question

I have a simple sidekiq job that I've set to run every 5 minutes using clockwork. What I'm worried about is that, if the job takes longer than 5 minutes, it will run again, which could cause some issues if it's running at the same time as the original job.

Is there a way to lock the clockwork so that it only runs if the previous running has completed?

Answer 1

One option is to create a file with the process PID when the job starts, and delete it when the job ends, and abort the job at the beginning if the PID file already exists.

Although this particular part of my app isn't using sidekiq, the idea is the same. Here's how my app does it: https://github.com/WikiEducationFoundation/WikiEduDashboard/blob/master/lib/data_cycle/constant_update.rb

Answer 2

Sidekiq allows you to query the queue through the API.

You could simply query the sidekiq queue when your task runs. Check to see if the queue and/or job is still there. If it is, do nothing. If it isn't, enqueue the job.

I don't know if your job sits in a separate queue, which would probably be best. That way you can just view the queue size and if it is 0 then start up another process.

Sidekiq API

Answer 3

I've used the Sidekiq Unique Jobs gem to accomplish this. It has various strategies for preventing duplicate jobs.

Answer 4

Sidekiq Enterprise includes support for unique jobs. Unique jobs are defined with an option in the worker class:

# app/workers/my_worker.rb
class MyWorker
  include Sidekiq::Worker
  sidekiq_options unique_for: 10.minutes

  def perform(...)
  end
end

They must be enabled in the Sidekiq initializer as well:

# config/initializers/sidekiq.rb
Sidekiq::Enterprise.unique! unless Rails.env.test?

The time window only applies while the job is running, e.g., if it is unique for 10 minutes and the job completes in 6 minutes then Sidekiq will immediately allow another job to be enqueued; you don't have to wait 4 more minutes for the uniqueness lock to be dropped.

The uniqueness constraint is enforced across all Sidekiq Enterprise nodes.

You may need to adjust your retries options because a failed job will continue to hold the lock for as long as it is in a retry state. (even if it is not actively executing)