We are running Laravel 7 and Horizon 4.3.5. Horizon runs with Supervisor.
We have 10 different queues configured, but workers responsible for one particular queue constantly dies without any output. After restarting Horizon, I can see these workers up and running for several seconds via top and ps commands. Then they are gone.
I checked supervisor's stdout_logfile: nothing suspicious there. I can see Jobs related to this queue are being processed successfully. Each worker processes exactly 2 jobs before crash.
I checked supervisor's stderr_logfile, but it's empty.
Laravel logs and failed_jobs table both are empty.
I even checked syslog, but nothing related there.
There are no problems with other queues at all. Only this particular queue keeps piling up: jobs are being pushed to queue by application, but never processed until I restart Horizon.
There are lot of free space on disk, free RAM, CPU usage is low.
Worker command: /usr/bin/php7.4 artisan horizon:work redis --delay=0 --memory=128 --queue=main --sleep=3 --timeout=1800 --tries=1 --supervisor=php01-Mexm:business
Turned out it was Out Of Memory problem. We had one job in this queue which caused crash.
Still not sure why logs were empty. Probably there wasn't enough memory to log anything.