Handling errors while working with code snippets

Question

I am running a bunch of code all at once in python by copying it from my editor and pasting it into python. This code includes nested for loops. I am doing some web scraping and the program quits at different times. I suspect that this is because it doesn't have time to load. I get the following error (once again - the program scrapes different amounts of text each time):

Traceback (most recent call last):
  File "<stdin>", line 35, in <module>
IndexError: list index out of range

First, what does line 35 refer to? Is this the place in the relevant inner for-loop?

Second, I think that the error might be caused by a line of code using selenium like this:

driver.find_elements_by_class_name("button")[j-1].click()

In this case, how can handle this error? What is some example code with either explicit waits or exception handling that would address the issue?

Answer 1

You can try your code and catch an IndexError exception like this:

try:
    # your code here
except IndexError:
    # handle the error here

An IndexError happens when you try to access an index of a list that does not exist. For example:

>>> a = [1, 2, 3]
>>> print(a[10])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

It's difficult to say how you should handle the error without more detail.

Answer 2

When working with code snippets, it's convenient to have them open in a text editor and either

• only copy-paste into a console the part you're currently working on so that all the relevant variables are in the local namespace that you can explore from the console, or
• copy-paste a moderate-to-large chunk as a whole while having enabled automatic post-mortem debugger calling, e.g. with Automatically start the debugger on an exception Activestate recipe or IPython's %pdb magic, or
• run a script as a whole under debugger e.g with -m pdb, IPython's %run or by using an IDE.

Answer 3

It means that [j-1] doesn't exist for a given value of j, possibly if j-1 exceeds the max number of elements in the list