Resque: don't enqueue redundant jobs?

Question

I have a set of Resque queues, arranged by priority. For all of the jobs on the lowest-priority queue, they have no arguments, and a later run will supersede any previous runs -- e.g. they are syncing data with an external service. So it makes no sense to create a backlog with multiple copies of these jobs -- running it three times in a row adds no value.

But during busy periods, we do get a backlog (on this queue only) since the workers are busy with higher-priority jobs.

Can I prevent this backlog by stopping redundant jobs from being enqueued in the first place?

Answer 1

Yes, there are Gems that are available to do this. (AFAICT, they work by hashing the job class with its list of arguments and maintaining a set of "keys already enqueued" separately in Redis.)

See https://github.com/resque/resque-loner and https://github.com/neighborland/resque_solo.