I am trying to extract data from a website that is structured in this matter using python and selenium:
<table>
<tbody>
<tr>
<td> text </td>
<td>
<td> text </td>
</td>
<td>
<td> text </td>
</td>
<tr>
<td> text </td>
<td>
<td> text </td>
</td>
<td>
<td> text </td>
</td>
This the code I have for the data extraction using selenium:
data=[]
for tr in driver.find_elements_by_xpath('//table[@id="pinnedtablepositionsTable"]//tr'):
tds =tr.find_elements_by_tag_name('td')
if tds:
data.append([td.text for td in tds])
print(data)
When I print the data I am only the first column of the for each row in the website. The table on the website is structured in a matter where the rows are nested inside of each other. Has any one ever dealt with a website that is structured in this format. I am open to other scrapers besides selenium I just like to use it for automation purposes.
I think the indentation is wrong for your program , It should be -