StaleElementReferenceException even when having explicit wait

Question

I've been trying to crawl data from the website AlgoExplorer. It has a table with pagnigation to store data. Even though I use an Explicit Wait for clicking a 'next' button, it still get StaleException. Here is a piece of my code, and an image of error:

for i in tqdm(range(5)): 
  page = driver.find_element_by_tag_name('tbody').find_elements_by_tag_name('a')
  for e in page:
    pages.append(e.text)
  WebDriverWait(driver, 5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '.pagination.next'))).click()

Of course, every variable has been declared and library has been imported.

Can you please explain me why I still have that exception? enter image description here

Answer 1

StaleElementReferenceException is thrown when an element is no longer attached to the page. My guess is that sequence looks like:

1. element has been found
2. html dom was rebuild - element can be found with used locator BUT it's not the element found in step 1)
3. js script tries to interact with element found in step 1) BUT it's not THE SAME element (it has the same attributes etc. but it is a different element in case of Selenium

You can try to verify my hypothesis by checking what DOM elements has been added/removed with https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver