I found an awesome bit of code online from this source:
https://webscraping.com/blog/Scraping-multiple-JavaScript-webpages-with-webkit/
that empowers me to scrape javascript-heavy sites with ease. However, some sites only fully load when there is mouse scrolling involved. How do I integrate mouse scrolling into this particular set of code? I have googled super alot for this and I found methods to read AJAX calls and stuff but I still would like to use this set of code as some websites have a ton of calls that are impossible to track. Hence I would rather use this set of code to just scrape the html.
Part of the code is here, the rest is in the link above.
class Render(QWebPage):
def __init__(self, urls, cb):
self.app = QApplication(sys.argv)
QWebPage.__init__(self)
self.loadFinished.connect(self._loadFinished)
self.urls = urls
self.cb = cb
self.crawl()
self.app.exec_()