Issue with extracting data from javascript generated html doc

25 views Asked by At

I'm trying to parse info from this page https://fem.encar.com/cars/detail/35902422?wtClick_index=187&conType=pctom The data I need is in the following part of html:

<span class="DetailSummary_num_graph__oN21B">
<span>82%</span>
</span>

I need to get this 82%.

I've saved html file with following function:

async def discount(folder):
    url = f"https://fem.encar.com/cars/detail/35902422?wtClick_index=187&conType=pctom"
    headers = {
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36",
    }

    async with aiohttp.ClientSession() as session:
        async with session.get(url=url, headers=headers) as response:
            data = await response.text()
            if not os.path.exists(folder):
                os.makedirs(folder)
            with open(f"{folder}\html.html", "w", encoding="utf8") as file:
                file.write(data)

However, the html doc saved doesn't have the info I need which I see on the browser. Please help me find this data in json or other type of files on this webpage

0

There are 0 answers