scrape google search results using python and google API and error 10060

20 views Asked by At

I have a csv file with some keywords. I want to collect all search results of all keywords. This code did go well till now. Python presents error 10060. I want to know if my IP is restricted? And how to fix this problem. Thank you very much. In fact, I have nothing to work towards right now. I'm trying to contact the google API but I haven't found the contact method yet. I have used API key, cx, one list of headers, cookies, time.sleep(60)

import requests
import csv
import time

def search_google(query, fname, isin, start_page=1, num_pages=2):
api_key = "****"
cx = "****"
headers = {'User-Agent': '****',
           'Referer': 'https://www.googleapis.com'}
cookies = {'Cookie': '***'}
results = []

for page in range(start_page, start_page + num_pages):
    start = (page - 1) * 10 + 1
    url = f"https://www.googleapis.com/customsearch/v1?q={query}&key={api_key}&cx={cx}&start={start}&num=10"
    try:
        response = requests.get(url, headers=headers, cookies=cookies)
        response.raise_for_status()  
        page_results = response.json()
        for item in page_results.get("items", []):
            item["fname"] = fname  
            item["isin"] = isin 
            results.append(item)
        time.sleep(0)
    except Exception as e:
        print(f"error: {e}")
        write_error(fname, isin)
        break

return results

Sorry, I'm not very good at using this text compiler, so please ignore the indentation problem.

0

There are 0 answers