I'm trying to use a node module in nightmare evaluate callback function but every time I run my code I get error which says html2json is not defined. I tried using both import and require to import my module but It didn't work. I also tried using closure to pass the module and return the callback function where I have use that specific module but it didn't work either. If I use this module outside of callback function it works perfectly fine. Any help on this is greatly appreciated.
import Nightmare from "nightmare";
import html2json from "html2json";
const nightmare = Nightmare({ show: false });
async function startScrap() {
let result = await nightmare
.goto("https://duckduckgo.com")
.evaluate(() => {
const html = html2json.html2json(
document.querySelector("#r1-0 a.result__a").innerHTML,
);
console.log(html);
return document.querySelector("#r1-0 a.result__a").href;
})
.end()
.then(function (result) {
console.log(result);
});
}
startScrap();
The error that I get everytime :
node:internal/process/promises:265
triggerUncaughtException(err, true /* fromPromise */); rogramming\Personal Projects\scrappers\animelist-scrapper\test.js"
^
ReferenceError: html2json is not defined
at fn (<anonymous>:6:26)
at javascript (<anonymous>:25:21)
at <anonymous>:40:3
at EventEmitter.electron.ipcRenderer.on (d:\Programming\Personal Projects\scrappers\animelist-scrapper\node_modules\electron\dist\resources\electron.asar\renderer\web-frame-init.js:36:30) s\scrappers\animelist-scrapper\node_modules\electron\dist\resources\electron.asar\renderer\web-frame-init.js:36:30)
at emitMany (events.js:147:13)
at EventEmitter.emit (events.js:224:7) {
code: -1
}
The function you pass in evaluate() is run in browser. Like, running the function by opening web page in your web browser. So, it does not have access to node modules.
Way to do it, is to select and extract the attributes of the elements you are interested in and then process it in then().
then() will have access to node modules.