Hi guys I was trying to extract data from https://newslab.malaysiakini.com/covid-19/en
import requests
from bs4 import BeautifulSoup
page = requests.get("https://newslab.malaysiakini.com/covid-19/en")
soup = BeautifulSoup(page.content, 'html.parser')
option_tags = soup.find(id="uk-grid uk-grid-small uk-width-auto uk-flex uk-flex-middle uk-flex-center")
patient_items = option_tags.find_all(class_="patient")
first = patient_items[0]
print(first.prettigy())
I cant extract the result seems like my html.parser cannot get the data like I see in the google console. Anyone can help on this?
The site makes a lot of requests after the initial requests to
https://newslab.malaysiakini.com/covid-19/en
. These additional links may have what you're looking for.This link probably has all the information you are looking for except the GPS coordinates. The locaiton is more difficult, they appear to be compiled into some javascript and data tags.
https://m5.malaysiakini.com/en/tag/covid-19?alt=json This contains a JSON format of all the stories on the google map/list. For example: