Expire for Laravel Jobs

Question

I parse some HTML pages and API endpoints, for example, every 5 minutes to track changes. For this purpose, I have created ParseJob where I do parsing and save changes to a database. ParseJob implements interface ShouldQueue and I have changed queue driver to Redis. In order to run the ParseJob on a regular basis, I have created ParseCommand and added it to schedule:

class ParseCommand extends Command
{
    protected $signature = 'application:my-parse-command';

    public function handle()
    {
        $this->dispatch(new ParseJob());
    }
}


class Kernel extends ConsoleKernel
{
    protected $commands = [
        Commands\ParseCommand::class
   ];

    protected function schedule(Schedule $schedule)
    {
           $schedule->command('application:my-parse-command')
            ->everyFiveMinutes();
    }
}

And the queue worker is started as a daemon to process the queue. So, every 5 minutes ParseJob is pushed to the queue and the queue worker is processing the job.

Sometimes queue worker process crashes, freezes or for other reasons is not working. But jobs every 5 minutes are pushed into the queue. After an hour of downtime, I have 12 jobs in the queue but they are for that time irrelevant because I do not need to parse 12 times at a certain time, I want just one parse job.

So I want to set TTL for a job that works like expire command in Redis. How to do that? Or maybe you have an alternative solution?

Answer 1

As far a i know it is not possible to set explicitly a Job expiration into Laravel queues. A solution could be setting an expires_at property within your ParseJob and check before executing:

class ParseCommand extends Command
{
    protected $signature = 'application:my-parse-command';

    public function handle()
    {
        $this->dispatch(new ParseJob(Carbon::now()->addMinutes(5)));
    }
}

then in your Job class

class ParseJob {

    protected $expires_at;

    public function __construct(Carbon $expires_at) {
       $this->expires_at = $expires_at;
    }

    public function handle() 
    {

        if(!Carbon::now()->gt($this->expires_at)) {
           // Parse data
        }

    }

}

Answer 2

At larvel 8/9 in job

MyJObClass

public function retryUntil(): Carbon
{
  return now()->addMinutes(10);
}

Laravel :: Queues #time-based-attempts