How to avoid getting HTTP Error 429: Too Many Requests with this code?

846 views Asked by At

How Can avoid getting HTTP Error 429: Too Many Requests with this code or handle it with this code? I have tried adding sleep.time(sec) but it doesn't work

import googlesearch
import pandas as pd

with open('Unknown.xlsx', "rb") as f:
    df = pd.read_excel(f)  # can also index sheet by name or fetch all sheets
    mylist = df['Short Code'].tolist()

try:
    from googlesearch import search
except ImportError:
    print("No module named 'google' found")
# to search
shortcode = int()
shortcode_list = mylist

for i in range(len(shortcode_list)):
    shortcode = shortcode_list[i]
    string = "text * to " + '"' + str(shortcode) + '"'
    print(string)
    query = string
    # time.sleep(1)
    for j in search(query, tld="co.in", num=10, stop=10, pause=2, country='US',
                    user_agent=googlesearch.get_random_user_agent(), verify_ssl=True):
        print(j)
1

There are 1 answers

0
raoyc On
  • Control your request limits.
  • Passing some normal browser User-Agent/cookies ... headers to your request.
  • Using frontend headless-chrome or chrome-extensions to crawl data.
  • Using proxy servers with http(s)-proxy on per request.
  • Buy some paid services such as https://serpstack.com/ .
  • ...