Elasticsearch-py bulk helper equivalent of curl with file

Question

I am looking to replicate the following command using the elasticsearch python client (and without using subprocess):

curl -s -XPOST "localhost:9200/index_name/_bulk" --data-binary @file

I have attempted to use the bulk helper without any luck:

es = Elasticsearch()

with open("file") as fp:
    bulk(
        client=es,
        index="index_name",
        actions=fp
    )

This results in type is missing errors.

The file, which is processed just fine when using curl, looks a bit like this:

{"index":{"_type":"someType","_id":"123"}}
{"field1":"data","field2":"data",...}
{"index":{"_type":"someType","_id":"456"}}
{"field1":"data","field2":"data",...}
...

Please note, I'd rather not change the contents of the file since I have around 21000 with the same format.

Answer 1

The actions parameter must take an iterable (not a file handle) that will iterate over the lines of your file, so you need to do it like this instead:

es = Elasticsearch()

def readbulk():
    for line in open("file"):
        yield line

bulk(
    client=es,
    index="index_name",
    actions=readbulk
)