I am using vladimir-yuldashev/laravel-queue-rabbitmq to connect in laravel & forge, with a RabbiqMQ message producer, Connection Lost/ broker Pipe

Question

I have setup with the following configuration.

  'derayah' => [
            'driver' => 'rabbitmq',
            'queue' => env('RABBITMQ_QUEUE', 'default'),
            'connection' => PhpAmqpLib\Connection\AMQPStreamConnection::class,

            'hosts' => [
                [
                    'host' => env('RABBITMQ_HOST', '127.0.0.1'),
                    'port' => env('RABBITMQ_PORT', 5672),
                    'user' => env('RABBITMQ_USER', 'guest'),
                    'password' => env('RABBITMQ_PASSWORD', 'guest'),
                    'vhost' => env('RABBITMQ_VHOST', '/'),
                ],
            ],
            'worker' => env('RABBITMQ_WORKER', 'default'),

            'options' => [
                'ssl_options' => [
                    'cafile' => env('RABBITMQ_SSL_CAFILE'),
                    'local_cert' => env('RABBITMQ_SSL_LOCALCERT'),
                    'local_key' => env('RABBITMQ_SSL_LOCALKEY'),
                    'verify_peer' => env('RABBITMQ_SSL_VERIFY_PEER', true),
                    'passphrase' => env('RABBITMQ_SSL_PASSPHRASE'),
                ],
                'queue' => [
                    'job' => App\Listeners\Derayah::class,
                ],
            ],
        ],

I am using Laravel Forge to create Deamon, and here is the command

queue:work' derayah --sleep=10 --daemon --quiet --delay=10 --tries=5 --queue='B_Transaction,S_Transaction,M_Transaction'

It is a stock market feed, which provides messages on if a particular person has done some transaction on a broker, and we save it on our system, It usually works fine, but sometimes it fails with following error

PhpAmqpLib\Exception\AMQPRuntimeException /artisan in ?
Lost connection: Broken pipe or closed connection

and then this keeps coming in logs

PhpAmqpLib\Exception\AMQPIOException /artisan in ?
stream_socket_client(): Unable to connect to tcp://213.184.187.103:5672 (Connection refused)

My Solution: I need to restart the QUEUE manullay from forge in order for this to connect again after a failure

My Findings: Its a no-ack setup, i need to find a way to restart the queue/connection again as it seems my queue gets stuck

Answer 1

The broker (RabbitMQ server) closed the connection for some reason. Or maybe it restarted and the connection in the worker is no longer valid.

The usual solution to avoid issues like this is to run processes under supervisord (http://supervisord.org/) or similar so that they are automatically restarted when an error happens.