I'm trying to get the src values for all images on Bing image search for a search term. I am using puppeteer for it. I wrote a selector to grab each image tag and it works in the Chrome DevTools. It, however, isn't working when I write it in the code-

const puppeteer = require("puppeteer");

(async () => {
    try{
        let url = `https://www.bing.com/images/search?q=cannabis`
        const browser = await puppeteer.launch({headless: false})
        const page = await browser.newPage()
        await page.goto(url)
        await page.waitForSelector("ul.dgControl_list li img.mimg")

        console.log(await page.evaluate(() => {
            Array.from(document.querySelectorAll("ul.dgControl_list>li img.mimg"), img => img.src)
        }))
    } catch(err){
        console.log("error - " + err)
    }
})()

I get the output as an object containing arrays of 10 items each in the devTools, but when I run it in the console through my code, it is undefined. How do I read this object?

1 Answers

1
Thomas Dondorf On Best Solutions

You are not returning any data from the page.evaluate call. To return the data you have to use the return statement or use the short syntax (as explained below):

console.log(await page.evaluate(() => {
    return Array.from(document.querySelectorAll("ul.dgControl_list>li img.mimg"), img => img.src)
}))

Explanation: Arrow function

The arrow function has two ways to write them. One is the short syntax, you can use it like this:

const func = () => 1; // func() will simply return 1

You can only put in one statement in there (which might call other statements though). Alternatively, you can use the long form:

const func = () => { return 1; }; // Same function as above

You can use variable declarations and any kind of code inside this function (just as in a normal function() { ... }, but this time you have to use return to return a value.

Therefore, as an alternative, you could also write this (short syntax):

console.log(await page.evaluate(
    () => Array.from(document.querySelectorAll("ul.dgControl_list>li img.mimg"), img => img.src)
))