Bullmq addBulk stuck on active

90 views Asked by At

I'm in the midst of setting up a nodejs background runner with Bullmq, hosted on vercel.com.

The setup uses Redis, and works in general fine. However, i had to do some cleanup of the code because of the Serverless function on Vercel, timed out, because of the max 300s (5m) time limit. And therefore I adjusted the code to use the addBulk instead of adding each individually, with a lookup to GraphQL for some extra data.

Works great locally, jobs running and completes as expected. But when I deploy to Vercel, and run the code again, the first job just sits in the active queue, and the rest are in the waiting queue, and nothing happens.

The worker also instantly goes to idle state (from taskforce.sh) I've tried what feels like everything, and can't find a solution.

This is the current code snippet that handles the worker, and what happens for each job.

The getVariant and updateVariant runs GraphQL queries

const settings = {
  connection,
  removeOnComplete: {
    count: 1500,
  },
  removeOnFail: {
    count: 7500,
  },
  metrics: {
    maxDataPoints: MetricsTime.ONE_WEEK,
  },
} satisfies WorkerOptions;

const updateJob = async (job: Job) => {
  const { sku, quantity } = job.data;

  try {
    const variant = await getVariant(sku);

    if (!variant) {
      const error = new Error(`Variant by SKU: ${sku} not found`);
      await job.moveToFailed(error, `SKU-${sku.toString()}`, true);

      throw error;
    }

    await updateVariant(variant, quantity);
    return job.moveToCompleted('DONE', `SKU-${sku.toString()}`, true);
  } catch (error: any) {
    await job.moveToFailed(error, `SKU-${sku.toString()}`, true);
    console.error('error', error.message);
  }
};

const worker = new Worker(queueName, updateJob, settings);

worker.on('active', (job) => {
  console.log(`Current active job with id ${job.id}`);
});

worker.on('stalled', async (jobId) => {
  console.error(`Job with id: ${jobId} stalled`);
});

worker.on('completed', async (job) => {
  console.log(`Completed job: ${job.id}, data: ${job.data}`);
});

worker.on('error', (error) => {
  console.error(`Job encountered an error: ${error.message}`);
});

worker.on('failed', (job, error) => {
  console.error(`Job failed with error: ${error.message}`);
});
0

There are 0 answers