Why am I receiving an Exception has occurred: keyerror 'original' using SerpApi Google Images scraper after running about ~40 searches?

Question

When I run the code below, I receive an Exception has occurred: keyerror 'original' error. I am able to successfully pull ~100 url links for each query I run and then it abruptly stops on the 40th query that I run for apparently no reason. Nothing looks wrong with the query that I pass through.

I'm trying to pass through a list of names of people that I have stored in multiple .txt files, then pull ~100 image URLs related to those people for each search query that I run. I thought it might be giving me an error because the name was improperly formatted or something, but there's nothing unusual with the format.

See below for an example of the names being passed through:

Carolyn B. Maloney
Tommy Tuberville
Cheri Bustos
Ritchie Torres
Rashida Tlaib
Kathy E. Manning
John Katko
Michael Cloud  
Hakeem S. Jeffries
Van Taylor <---- Fails here
Elijah Crane...

See below for a segment of the Python I'm running:

def get_image_urls(query, key):
    params = {
      "q": query,
      "tbm": "isch",
      "ijn": "0",
      "api_key": key
    }

    search = GoogleSearch(params)
    results = search.get_dict()
 
    image_urls = [image["original"] for image in results["images_results"]]
    return image_urls

image_urls_dict = {}
for file in file_names:
    with open(file, 'r') as f:
        for person in f:
            
            name = person.strip()
            print(name)
            image_urls_dict[name] = get_image_urls(name, api_key)

Answer 1

You need catch and deal with all possible exception, such as pass the value which don't have key "original", or retry it. Or set a breakpoint at the "pass" line to watch what actually bugs happen to "results".

Example code:

def get_image_urls(query, key):
    params = {
      "q": query,
      "tbm": "isch",
      "ijn": "0",
      "api_key": key
    }

    # search = GoogleSearch(params)
    # results = search.get_dict()

    results = {"images_results":[{"original":"B. Maloney"},{"original":"Tuberville"},{"missing_original":"Bustos"}]}
    image_urls = []
    for image in results["images_results"]:
        try:
            image_urls.append(image["original"])
        except KeyError:
            pass #or you can retry to get correct results.
    return image_urls