I’m using puppeteer to do some crawling on cloudflare worker and I was testing it on an API model. After I finished the code and put it on a queue it starts to slow down the queue to always exceed cpu usage. The logs in realtime take forever to update and the queue stops processing new messages until the message that exceeded cpu finish.
At first I thought it was my code, so I commented some things and got to a point when I just launch the browser log something and then close the browser. I also tried to explicit acknowledge messages and added a return at the end just to make sure some kind of “keep executing forever” error wasn’t the problem. The same response Anytime I remove puppeteer code the queue executes instantly with the logs. If I try to launch the browser, it exceeds cpu. The config is the same as the docs example with some name changes.
After nothing worked I tried to use the same exact code from cloudflare documentation about how to do a crawler with puppeteer on a queue. This one:
https://github.com/cloudflare/queues-web-crawler/tree/master
The only change I’ve done to the code was changing the “kv put” code to a simple console.log. The behavior was the same.
Does anyone have any idea why is this happening or had the same problem? Can you run the cloudflare doc code without those problems?