Click button with requests_html via javasript and retrieve the data in a loop

Question

Click Button with requests_html via Javasript and get the Data in Loop

I am trying in vain to run javascript on a website to fetch the information.

I want to click a button with javascript after loading the website with requests_html and read the web page.

This is what I want to happen:

1. web page load
2. trigger button with javascript
3. read the new data
4. execute point 2-3 times as often as you want.

The button on the website looks like this:

<a href="#" class="next-button" data-v-52c9cec4="">Mehr <img src="https://d18yn9dcojt05d.cloudfront.net/apps/webdevs/unified-search-frontend/_nuxt/arrow-right-blue.2cb64467.svg" width="24" height="24" alt="Next" loading="lazy" data-v-52c9cec4=""></a>

My Python program:

from requests_html import HTMLSession
session = HTMLSession()
r = session.get('https://www.thisistheurl.com')
session = HTMLSession()
r = session.get(URL)
script = """
    () => {
        document.getElementsByClassName('next-button')[0].click();
    }
"""
r.html.render(sleep=5, script=script)
print(r.html.html)

With this code I do not get the new data. I have also tried sleep, reload=False and keep_page=True.

Can someone help me here what I am doing wrong?

Sleep, reload=False and keep_page=True Also inserted a sleep in the javascript without success.