I'm Writing a CODE Algorithm BELOW:
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import requests
import time
browser =webdriver.Firefox(executable_path=r'C:/path/geckodriver.exe')
browser.get('https://brainly.com.br/app/ask?entry=hero&q=15+12')
WebDriverWait(browser,10).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,'a[href*="/tarefa"]')))
html=browser.page_source
#html = browser.execute_script("return document.documentElement.outerHTML")
p=[]
soup=BeautifulSoup(html,'html.parser')
for link in soup.select('div.sg-actions-list__hole > a[href*="/tarefa"]'):
ref=link.get('href')
rt = ('https://brainly.com.br'+str(ref))
p.append(rt)
g=[]
for url in p:
r = requests.get(url).text
time.sleep(10)
rs= BeautifulSoup(r,'html.parser')
df = rs.select('div > h1')
print(df)
HTML
I need to filter:
<div class="brn-content-image">
<h1 class="sg-text sg-text--large sg-text--regular">
Tinha 12 cruzeiros e gastei 15 cruzeiros esta situação pode ser representada por a 15 - 12 B 12 + 15 C 12 - 11 e D - 12 + 15
</h1>
</div>
AND
<div class="brn-content-image">
<h1 class="sg-text sg-text--large sg-text--regular">
[(-15)12]12:(-15)142?
</h1>
</div>
At Variable r
I am storing the source code for 8 URLS, ie 8 Source Code which I am not sure is the problem that I cannot filter it with.
Using Code to Filter HTML:
rs= BeautifulSoup(r,'html.parser')
df = rs.select('div > h1')
print(df)
What mistake am I making?.
Instead of using
Selenium
, you can call theAPI
directly which is rendering theJavaScript
usingXHR
request.Output:
Output: