I have written a Python code to scrape from a website for certain years. To this end, I have created a for loop to iterate over the years and use the string to navigate to the website. See below:
from selenium import webdriver
from bs4 import BeautifulSoup
from urllib.parse import urljoin
import pandas as pd
URL = 'url'
for year in range(2012,2021):
type(year)
stryear = str(year)
driver.get(URL + stryear)
content = driver.page_source
soup = BeautifulSoup(content,features="lxml")
#Rest of code
For some reason, the first iteration is all fine. When running the debug mode, I see that in the second iteration, the variable year is an integer with value 2013 (as expected). The variable stryear however, gets type {Resultset: 0} [], with source {SoupStrainer} 2013|{}.
Obviously, I am not declaring stryear anywhere else in the code. I need stryear to be a string containing the year. Why is the variable stryear not behaving as I expected?
Check your imports. Most probably
str()
was either imported locally from Soup library or gotmonkey patched
by it globally.