Using puppeteer page.evaluate to return a value that I can use outside of the browser context

Question

I am using page.evaluate to record each CLS entry on the browser and I am trying to save it to a variable that I can use outside the browser context. I am doing this as follows:

page.exposeFunction('getCLSEntries', list => {
  //console.log("list:", list);
  let clsList = JSON.parse(list);
  return clsList;
});
console.log("CLS List:", clsList);

let cls = new Map();
while (scrollCount < maxScrolls) {
    //console.log("Function exposed"
    
    cls[scrollCount] = await page.evaluate(() => {
        console.log("reportCLSInstances called");
        let clsEntries = [];
        let clsInstances = [];
        const internalCLSInstances = (list) => {
            clsEntries = list.getEntries();
            const entries = JSON.stringify(clsEntries);
            window.getCLSEntries(entries).then((value) => {
                clsInstances.push(value);
                console.log("value: ", value);
            });
            console.log("CLS Instances 1:", clsInstances);
        };
        const observer = new PerformanceObserver(internalCLSInstances);
        observer.observe({type: 'layout-shift', buffered: true});
        console.log("CLS Instances 2:", clsInstances);
        return clsInstances;
    });
    console.log("CLS: ", cls);

However, each return of page.evaluate gives me [], even when the array is populated in the log statement just before the return. What am I doing wrong?

When I replace the return value of page.evaluate to ["hello"], it populates my map as expected.

Answer 1

You have two levels of asynchronous code between the empty clsInstances that's currently returned and the version that's eventually populated but never seen:

• the callback passed into PerformanceObserver is called in some turn of the event loop after the function passed to page.evaluate() has returned the initial empty array
• the call to getCLSEntries is asynchronous (returning a promise), making the synchronous internalCLSInstances callback also not actually complete by the time it returns.

The answer to this is typically to wrap the whole thing in a Promise and resolve it where the work is actually complete.

There's a lot of lines in here that I'm assuming were added while debugging (like I'm not sure of the reason for passing back to node code in getCLSEntries except maybe for logging?), but assuming everything is necessary:

    cls[scrollCount] = await page.evaluate(() => {
      return new Promise(resolve => {
        console.log("reportCLSInstances called");
        let clsEntries = [];
        let clsInstances = [];
        const internalCLSInstances = (list) => {
            clsEntries = list.getEntries();
            const entries = JSON.stringify(clsEntries);
            window.getCLSEntries(entries).then((value) => {
                clsInstances.push(...value);
                console.log("value: ", value);
                resolve(clsInstances);
            });
            console.log("CLS Instances 1:", clsInstances);
        };
        const observer = new PerformanceObserver(internalCLSInstances);
        observer.observe({type: 'layout-shift', buffered: true});
        console.log("CLS Instances 2:", clsInstances);
      });
    });

(This snippet also spreads the value array, assuming the goal is an array of layout-shift entries, not an array where the first element is an array of layout-shift entries)