I have been experiencing this issue from a long time now. I have a web scraper on a Windows VM and I have it set to run every few hours. It works most of the time but a lot of times Puppeteer just opens this page and not the site or page I want to open.
Why does that happen and what can be the fix for this?
A simple reproduction for this issue can be this code
import puppeteer from 'puppeteer'
import { scheduleJob } from 'node-schedule';
async function run() {
const browser = await puppeteer.launch({
headless: false,
executablePath: chromePath,
defaultViewport: null,
timeout: 0,
args: ['--no-sandbox', '--start-maximized'],
});
const page = await browser.newPage();
await page.evaluateOnNewDocument(() => {
Object.defineProperty(navigator, 'webdriver', {
get: () => false,
});
});
await page.goto('https://aliexpress.com', {
waitUntil: 'networkidle0',
timeout: 0,
});
}
run();
scheduleJob('scrape aliexpress', `0 */${hours} * * *`, run);