Nightmare do not scrape Amazon price information

Question

I am building an Amazon price tracker, and using node.js with the module nightmare for web scraping.

This is the Amazon page that I want to scrape information from: https://www.amazon.in//dp/B0BDKD8DVD/

My code just returns a NULL value instead of returning the price of the product.

This is My app.js code

const express = require("express")
const parser = require("./parser")

const app = express();

app.listen(3000, () => {
    console.log("listening on port 3000")
})

app.get("/", (req, res) => {
   const ans = parser();
    res.send(ans)
})

And this is my parser.js code

const nightmare = require("nightmare")();

async function checkprice() {

    const priceString = await nightmare
        .goto("https://www.amazon.in/Apple-AirPods-Pro-2nd-Generation/dp/B0BDKD8DVD/ref=sr_1_5")
        .wait(".a-offscreen")
        .evaluate(() => document.getElementsByClassName("a-price-whole").innerText)
        .end
    const priceNumber = parseFloat(priceString)
    console.log(priceNumber)
    return priceNumber

};

module.exports = checkprice;

This is returning NaN and not price. Any advice could be really helpful. Thank you.

Answer 1

Your element return Undefined

document.getElementsByClassName("a-price-whole").innerText

Correct one would be

document.getElementsByClassName("a-price-whole")[0].innerText

Take note that their 6 div with this Class Name and that return new line '26,600\n.'

Answer 2

If you want to get more information from the product page, it is very difficult to pick the right selectors and maintain your parser in the long run. Amazon (or any other website) often changes selectors or DOM tree structures by adding or deleting elements. Instead, to get Amazon's (or other popular e-commerce websites) product info you can use simple (and free) NPM package: ecommerce-scraper-js.

Usage example:

import { amazon } from "ecommerce-scraper-js";

amazon
  .getListingInfo("https://www.amazon.in/Apple-AirPods-Pro-2nd-Generation/dp/B0BDKD8DVD/ref=sr_1_5")
  .then((result) => console.dir(result, { depth: null }));

Output example:

{
   "title":"Apple AirPods Pro (2nd Generation) ​​​​​​​",
   "link":"https://www.amazon.in/Apple-AirPods-Pro-2nd-Generation/dp/B0BDKD8DVD/ref=sr_1_5",
   "productOverview":{
      "Brand":"Apple",
      "Model Name":"Apple AirPods Pro (2nd generation) ​​​​​​​",
      "Colour":"White",
      "Form Factor":"In Ear",
      "Connectivity Technology":"Bluetooth 5.3"
   },
   "features":[
      "Active Noise Cancellation reduces unwanted background noise",
      "Adaptive Transparency lets outside sounds in while reducing loud environmental noise",
      "Personalised Spatial Audio with dynamic head tracking places sound all around you"
   ],
   "reviewsInfo":{
      "rating":4.4,
      "reviewsAmount":1330,
      "byFeature":{
         "Bass quality":"4.2",
         "Noise cancellation":"4.1",
         "Sound quality":"4.1",
         "Battery life":"3.9"
      },
      "reviews":[
         {
            "name":"Sagara",
            "avatar":"https://images-eu.ssl-images-amazon.com/images/S/amazon-avatars-global/default._CR0,0,1024,1024_SX48_.png",
            "rating":5,
            "summary":"Best combination with iphone and mac",
            "dateAndPlace":"Reviewed in India on 25 May 2023",
            "bage":"Verified Purchase",
            "review":"Amazingly good functionality. Clear calls, others are able to hear you well - it does the primary stuff really well. Alternate between iphone and mac calls, noise cancellation - reduces by a lot. Only quirk is I have a feeling always that it's gonna fall down now; however it has never done so yet. I think it's my ear peculiar, as I have tried
all combinations of supplied eartips."
         },
         ... and other reviews
      ]
   }
}

Disclaimer: I'm author of this package