I am hitting an elastic database and I want to return results one by one using scroll(), which requires me to fetch a scroll_id using the search() method.

The result size can range from 0-40000, that's why I can't use search() for the whole thing.

I didn't notice the problem when there were a large number of hits, but when there is only one hit, the scroll doesn't return anything.

query = {"query":{"match":{"message": "Techtotic"}}, "sort": [{"@timestamp":"asc"}]}

scrollid = es.search(index="ts-shanks*", size=2, scroll="50m", body=query)

hits = scrollid["hits"]["total"]

scrollid = scrollid["_scroll_id"]

for i in range(hits):

    entry = es.scroll(scroll_id=scrollid, scroll="1m", filter_path=req_data)

    try:
        print(entry["hits"]["hits"][0]["_source"]["message"])

    except Exception:
        print(json.dumps(entry, indent=4))

The output displays the matched hit in the print corresponding to the '''search()''' but not in the one with the scroll().

Also, no matter how many hits, we get the last output of the scroll always triggers the except block because it's empty.

I heard that search_type="scan" would fix it, but it seams that it has been deprecated.

0 Answers