how to disable page query in Spring-data-elasticsearch

Question

I use spring-data-elasticsearch framework to get query result from elasticsearch server, the java code like this:

public void testQuery() {
    SearchQuery searchQuery = new NativeSearchQueryBuilder()
        .withFields("createDate","updateDate").withQuery(matchAllQuery()).withPageable(new PageRequest(0,Integer.MAX_VALUE)).build();
    List<Entity> list = template.queryForList(searchQuery, Entity.class);
    for (Entity e : list) {
        System.out.println(e.getCreateDate());
        System.out.println(e.getUpdateDate());
    }
}

I get the raw query log in server, like this:

{"from":0,"size":10,"query":{"match_all":{}},"fields":["createDate","updateDate"]}

As per the query log, spring-data-elasticsearch will add size limit to the query. "from":0, "size":10, How can I avoid it to add the size limit?

Answer 1

You don't want to do this, you could use the findAll functionality on a repository that returns an Iterable. I think the best way to obtain all items is to use the scan/scroll functionality. Maybe the following code block can put you in the right direction:

    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withQuery(QueryBuilders.matchAllQuery())
            .withIndices("customer")
            .withTypes("customermodel")
            .withSearchType(SearchType.SCAN)
            .withPageable(new PageRequest(0, NUM_ITEMS_PER_SCROLL))
            .build();
    String scrollId = elasticsearchTemplate.scan(searchQuery, SCROLL_TIME_IN_MILLIS, false);
    boolean hasRecords = true;
    while (hasRecords) {
        Page<CustomerModel> page = elasticsearchTemplate.scroll(scrollId, SCROLL_TIME_IN_MILLIS, CustomerModel.class);
        if (page != null) {
            // DO something with the records
            hasRecords = (page.getContent().size() == NUM_ITEMS_PER_SCROLL);
        } else {
            hasRecords = false;
        }
    }