How to push data to elasticsearch from dataframe in python

966 views Asked by At

I am trying to use elastic search libraries like pyelasticsearch and elasticsearch I am not getting any method where i can pass dataframe and that method will load data frame data to elastic search.

I am trying a this code:

for i,df in enumerate(csvfile): 
print (i)
records=df.where(pd.notnull(df), None).T.to_dict()
list_records=[records[it] for it in records]
print(list_records)
try :
    es.bulk_index("call_demo_data1","tweet",list_records)
except :
    print ("error!, skiping some tweets sorry")
    pass

where csvfile is my dataframe where my all data is present. but I am getting following error

'str' object has no attribute 'where'

I have used recommendation in comments

Now that problem is solved I am getting this error while bulk loading

I am using above method to load data elastic search I was facing problem so here is the link to the question I posted earlier

Here is the code I am using now :

records= csvfile.T.to_dict()
list_records=[records[it] for it in records]
#print(list_records)
es.bulk_index("call_demo_data1","tweet",list_records)

Error I am getting is :

too many values to unpack (expected 2) 

this error is coming while bulk indexing. csvfile in above code is a data frame. I am using this liabrary pyelasticsearch

This is the error traceback Traceback

0

There are 0 answers