Unable to click the Explore text in Google Trends site using Selenium Python

Question

I need to get Google Trends data for past 30 days, past 12 months like that for the purpose of my current project. I'm able to get daily data using Google Trends API. The package has another API called InterestsOverTime but Google is blocking any kind of automation effort when I try to access Explore Page Using this API, web crawling or even Selenium.

So I decided to go to the main page and then click on the Explore page But I'm failing to do so using Selenium. I'm very new to Selenium so please help me out here

This is the code I'm using

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as ES
chrome_options = Options()
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--window-size=1920x1080")

driver = webdriver.Chrome(options=chrome_options, keep_alive=True)
url = "https://trends.google.com/trends/"
driver.get(url)
explore_button = WebDriverWait(driver,10).until(
ES.visibility_of_element_located((By.XPATH,"//span[contains(text(),'Explore')]"))
)
explore_button.click()

I tried multiple ways to access including using wrapper APIS from Google Trends API and trying to access Explore Page using python web crawling and selenium but was not able to get it. Even trying to touch the text Explore using Selenium is not working

Answer 1

Identify the button element uniquely by following xpath.

//button[contains(@jsaction,'clickmod')][.//span[contains(text(),'Explore')]]

Code:

explore_button = WebDriverWait(driver,10).until(
ES.visibility_of_element_located((By.XPATH,"//button[contains(@jsaction,'clickmod')][.//span[contains(text(),'Explore')]]"))
)
explore_button.click()

Answer 2

Issue is with this XPath expression: //span[contains(text(),'Explore')]

First issue is, it is locating 3 web elements. Second issue is you are targeting the span element you should be targeting button element.

Just change the XPath expression as below:

(//span[text()='Explore']//parent::button)[1]

Explanation of XPath:

• //span[text()='Explore']: This part of the expression selects any <span> element in the document that has the exact text content "Explore".

• //parent::button: This part selects the parent of the previously selected <span> element, but only if the parent is a <button> element.

• [1]: Finally, the [1] at the end limits the result to the first matching element in case there are multiple elements that match the criteria.