Search twitter historically for relevant tweets

288 views Asked by At

I want all tweets related to a particular an event how can I do it? A rough search suggests it is not possible. There are some paid solutions though. https://gnip.com/sources/twitter/historical/ http://keyhole.co/ Is there any workaround?

Intent: I am doing a project which gets me all tweets related to a particular event. Therefore, I search using some keywords. Though this is not accurate I later on can use some classifier to filter the noise.

2

There are 2 answers

8
Shruti Srivastava On BEST ANSWER

You can use a bs4 to parse through your tweets and store the keywords in some file or database

I am doing a similar project and what i have done is cretaed a regex to match the keywords.

6
Alex Huszagh On

Through the public API, it is not possible. However, within Twitter itself, you can search for old Tweets, so it is possible there is a workaround using a click/search bot or web scraping.

I would recommend this post (and the comments) here: https://stackoverflow.com/a/24246840/4131059

The limit on their API (as I myself have done) is 2 years. For their search on their webpage, it is unlimited.

EDIT: It's not code: you can automate harvesting data using search queries via their site, but you cannot use their API to do so. I'll give you how to use the search queries to get what you want:

It's not code: you need manually harvest the data using their search service on their website.

This is how you use it: Keywords: from, since, until

Example: code from:tenderlove since:2010-01-01 until:2012-12-31

Grabs all tweets from the user @tenderlove (a well-known coder) that has the word "code" in the tweet from January 1st, 2010 to December 31st, 2012.

Now if you try this with any API query, it will give you no tweets, because it is outside their date range.