I have a big json file of tweets(around 5GB). I am having memory error. So, I decided to parse the data. I found ijson package. I am having such an error:
import ijson
parser = ijson.parse(tweets_data_path )
tweets_data = []
f = open(tweets_data_path, "r")
objects = ijson.items(f, 'other_config.item')
for line in objects:
try:
tweet = json.loads(line)
tweets_data.append(tweet)
except:
continue
"No module named ijson" I am fairly new to Python, I looked at the source file of the package. But I could bout quite get what the requirements are. Any help will be appreciated.
ijson
is an external package that is not included with the regular python libraries. You need to installijson
yourself first. Look into using something like pip which is a package manager for python. Once installed you can installijson
through the terminal like this: