trouble getting page data only during iteration with pyppeteer

23 views Asked by At

I'm using pyppeteer to load a table from a url, and then I need to click on each record in the table so additional information will be present.

this is my code:

await page.waitForXPath('/html/body/div[1]/div[2]/div[2]/div[2]/div[1]/div/div[3]/table/tbody')
rows = await page.xpath('/html/body/div[1]/div[2]/div[2]/div[2]/div[1]/div/div[3]/table/tbody/tr')

for row in rows:
    try:
        await row.click()
        await asyncio.sleep(1)
        image_element = await page.waitForXPath('/html/body/div[1]/div[2]/div[3]/div[2]/div[1]/div[2]/div[2]/div[3]/div[2]/div/img',
                                                        {'timeout': 100000}, present=True)
        image_link = await page.evaluate('(img) => img.src', image_element)
        print(f"Image Link: {image_link}")
    except Exception as e:
        print(f"Error in iteration: {e}")

The problem is, that if I have 4 records for example, I will manage to get the first 3 image links, and the 4th one will always fail.

BUT When I try to get the link for the last record alone (without getting the other records) it works.

Any ideas how to fix it?

Thanks!

0

There are 0 answers