I'm scraping a website weather.com, I want to click on an html element to fetch data and show it, then I scrape that data.
But it doesn't always work for me, most of the time the click doesn't happen and I get an error trying to scrape those elements
await page.waitFor('#twc-scrollabe > table > tbody > tr:nth-child(1)')
await page.click('#twc-scrollabe > table > tbody > tr:nth-child(1)')
What should I do to ensure that the click happens?
Try specifying you want the element to be visible with
await page.waitFor('#twc-scrollabe > table > tbody > tr:nth-child(1)', { visible:true })
(do you notice the{visible:true}
option?)because the element could be in the page but not clickable when
waitFor
finds it.More details
waitFor
with a selector you're usingwaitForSelector
under the hood (see the docs)waitForSelector
has a series of options (see the docs)visible
, the docs sayLast but not least: @FeliFong asked you something more about your problem because you're not providing enough info about it. You could
It's useful for us but mostly for you because I tried to reply you... but I can't know if the solution I provided suite your needs... If I had the chance of checking it with a repo of your, I would have been sure I had solved the problem
[UPDATE] I fixed the issue on the repository you provided me, accept my PR on GitHub.
What I've done: I made some tests and I don't know what the hell "blocks" (or, better, stops from waiting) Puppeteer in that site... but it doesn't matter, this is my code