Im using selenium to scarpe some data from Search/Lastest from Twitter. 1 small problem is that, when they showing ad, I cant filter them out. I have some way to do that, but struggle and cant find the solution.
- Ad tweet have no time. I tried this, but it took too long.
try{
WebElement time = article.findElement(By.xpath(TIME_XPATH));
}catch (Exception e){
e.printStackTrace();
return true;
}
return false;
- Ad tweet have view count. Didn't try, but normal tweet dont have view count, so I dont think that gonna work.
- Ad tweet dont have the keyword text. But then, if i filter by that way, i cant get replies tweet, like replies @ElonMusk.
Any insights or suggestions on resolving this issue would be greatly appreciated. Thank you!