How to integrate mouse scrolling in a Javascript web scraper?

61 views Asked by At

I found an awesome bit of code online from this source:

https://webscraping.com/blog/Scraping-multiple-JavaScript-webpages-with-webkit/

that empowers me to scrape javascript-heavy sites with ease. However, some sites only fully load when there is mouse scrolling involved. How do I integrate mouse scrolling into this particular set of code? I have googled super alot for this and I found methods to read AJAX calls and stuff but I still would like to use this set of code as some websites have a ton of calls that are impossible to track. Hence I would rather use this set of code to just scrape the html.

Part of the code is here, the rest is in the link above.

class Render(QWebPage):
    def __init__(self, urls, cb):
        self.app = QApplication(sys.argv)
        QWebPage.__init__(self)
        self.loadFinished.connect(self._loadFinished)
        self.urls = urls
        self.cb = cb
        self.crawl()
        self.app.exec_()

0

There are 0 answers