Elasticsearch - "could not read the current timestamp”

Question

I am using Amazon Opensearch to search for profiles index. I get the following in application logs for a query that is using multi_match.

Amazon OpenSearch Version - Elasticsearch 6.4 Service software version - Elasticsearch_6_4_R20230928-P1 (latest)

Could someone please help?

[2023-09-19T17:07:52,876][DEBUG][o.e.a.s.TransportSearchAction] [O1onMaP] [xxxxxx][0], node[a24qYwKTTUO6yFcWf8QRKg], [P], s[STARTED], a[id=50vjfxaPSuqAJSwQVbqtNA]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[xxxxxx], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false], types=[profile], routing='null', preference='null', requestCache=false, scroll=null, maxConcurrentShardRequests=15, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, source={"suggest":{"suggestedKeyword":{"text":"Frenny","phrase":{"field":"text_all","size":1,"real_word_error_likelihood":0.95,"confidence":1.0,"separator":" ","max_errors":0.5,"gram_size":2,"force_unigrams":true,"token_limit":10,"collate":{"query":{"source":{"bool":{"filter":[{"range":{"lastLogin":{"from":"now-6M","include_lower":true,"include_upper":true,"to":null}}},{"terms":{"motherTongueId":["3","1"]}},{"term":{"gender":"m"}}],"must":{"multi_match":{"fields":["name^12","aboutMe^4","occupation^4","text_all^1"]__PATH__ charset=UTF-8"}},"prune":false}}}}}}]
RemoteTransportException[[a24qYwK][__IP__][__PATH__[__PATH__]]]; nested: QueryShardException[failed to create query: {
  "bool" : {
    "must" : [
      {
        "multi_match" : {
          "query" : "french",
          "fields" : [
            "aboutMe^4.0",
            "name^12.0",
            "occupation^4.0",
            "text_all^1.0"
          ],
          "type" : "best_fields",
          "operator" : "AND",
          "slop" : 0,
          "prefix_length" : 0,
          "max_expansions" : 50,
          "tie_breaker" : 0.0,
          "zero_terms_query" : "NONE",
          "auto_generate_synonyms_phrase_query" : true,
          "fuzzy_transpositions" : true,
          "boost" : 1.0
        }
      }
    ],
    "filter" : [
      {
        "range" : {
          "lastLogin" : {
            "from" : "now-6M",
            "to" : null,
            "include_lower" : true,
            "include_upper" : true,
            "boost" : 1.0
          }
        }
      },
      {
        "terms" : {
          "motherTongueId" : [
            "3",
            "1"
          ],
          "boost" : 1.0
        }
      },
      {
        "term" : {
          "gender" : {
            "value" : "m",
            "boost" : 1.0
          }
        }
      }
    ],
    "adjust_pure_negative" : true,
    "boost" : 1.0
  }
}]; nested: ElasticsearchParseException[could not read the current timestamp]; nested: IllegalArgumentException[features that prevent cachability are disabled on this context];
Caused by: [__PATH__] QueryShardException[failed to create query: {
  "bool" : {
    "must" : [
      {
        "multi_match" : {
          "query" : "french",
          "fields" : [
            "aboutMe^4.0",
            "name^12.0",
            "occupation^4.0",
            "text_all^1.0"
          ],
          "type" : "best_fields",
          "operator" : "AND",
          "slop" : 0,
          "prefix_length" : 0,
          "max_expansions" : 50,
          "tie_breaker" : 0.0,
          "zero_terms_query" : "NONE",
          "auto_generate_synonyms_phrase_query" : true,
          "fuzzy_transpositions" : true,
          "boost" : 1.0
        }
      }
    ],
    "filter" : [
      {
        "range" : {
          "lastLogin" : {
            "from" : "now-6M",
            "to" : null,
            "include_lower" : true,
            "include_upper" : true,
            "boost" : 1.0
          }
        }
      },
      {
        "terms" : {
          "motherTongueId" : [
            "3",
            "1"
          ],
          "boost" : 1.0
        }
      },
      {
        "term" : {
          "gender" : {
            "value" : "m",
            "boost" : 1.0
          }
        }
      }
    ],
    "adjust_pure_negative" : true,
    "boost" : 1.0
  }
}]; nested: ElasticsearchParseException[could not read the current timestamp]; nested: IllegalArgumentException[features that prevent cachability are disabled on this context];
    at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:321)
    at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:304)
    at org.elasticsearch.search.suggest.phrase.PhraseSuggester.innerExecute(PhraseSuggester.java:122)
    at org.elasticsearch.search.suggest.phrase.PhraseSuggester.innerExecute(PhraseSuggester.java:53)
    at org.elasticsearch.search.suggest.Suggester.execute(Suggester.java:39)
    at org.elasticsearch.search.suggest.SuggestPhase.execute(SuggestPhase.java:64)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:95)
    at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:324)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:357)
    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333)
    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329)
    at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: ElasticsearchParseException[could not read the current timestamp]; nested: IllegalArgumentException[features that prevent cachability are disabled on this context];
    at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:60)
    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:318)
    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:331)
    at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:459)
    at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:466)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:269)
    at org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:474)
    at org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:441)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:269)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:51)
    at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68)
    at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:316)
    ... 18 more
Caused by: java.lang.IllegalArgumentException: features that prevent cachability are disabled on this context
    at org.elasticsearch.index.query.QueryShardContext.failIfFrozen(QueryShardContext.java:357)
    at org.elasticsearch.index.query.QueryShardContext.nowInMillis(QueryShardContext.java:391)
    at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:58)
    ... 29 more
[2023-09-19T17:07:52,876][DEBUG][o.e.a.s.TransportSearchAction] [O1onMaP] All shards failed for phase: [query]
ElasticsearchParseException[could not read the current timestamp]; nested: IllegalArgumentException[features that prevent cachability are disabled on this context];
    at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:60)
    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parseToMilliseconds(DateFieldMapper.java:318)
    at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.isFieldWithinQuery(DateFieldMapper.java:331)
    at org.elasticsearch.index.query.RangeQueryBuilder.getRelation(RangeQueryBuilder.java:459)
    at org.elasticsearch.index.query.RangeQueryBuilder.doRewrite(RangeQueryBuilder.java:466)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:269)
    at org.elasticsearch.index.query.BoolQueryBuilder.rewriteClauses(BoolQueryBuilder.java:474)
    at org.elasticsearch.index.query.BoolQueryBuilder.doRewrite(BoolQueryBuilder.java:441)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:269)
    at org.elasticsearch.index.query.AbstractQueryBuilder.rewrite(AbstractQueryBuilder.java:51)
    at org.elasticsearch.index.query.Rewriteable.rewrite(Rewriteable.java:68)
    at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:316)
    at org.elasticsearch.index.query.QueryShardContext.toQuery(QueryShardContext.java:304)
    at org.elasticsearch.search.suggest.phrase.PhraseSuggester.innerExecute(PhraseSuggester.java:122)
    at org.elasticsearch.search.suggest.phrase.PhraseSuggester.innerExecute(PhraseSuggester.java:53)
    at org.elasticsearch.search.suggest.Suggester.execute(Suggester.java:39)
    at org.elasticsearch.search.suggest.SuggestPhase.execute(SuggestPhase.java:64)
    at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:95)
    at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:324)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:357)
    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:333)
    at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:329)
    at org.elasticsearch.search.SearchService$3.doRun(SearchService.java:1019)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.IllegalArgumentException: features that prevent cachability are disabled on this context
    at org.elasticsearch.index.query.QueryShardContext.failIfFrozen(QueryShardContext.java:357)
    at org.elasticsearch.index.query.QueryShardContext.nowInMillis(QueryShardContext.java:391)
    at org.elasticsearch.common.joda.DateMathParser.parse(DateMathParser.java:58)
    ... 29 more

I tried tuning the query and come up with this one. But nothing seems to be working.

Answer 1

If you are indeed on 6.4, it looks like a regression of a bug that was supposed to be fixed all the way back in 5.0.1. The workaround would be to move your range query from filter into the must clause. Since now changes every millisecond, you are not gaining anything from placing it into filter context.

Answer 2

The IllegalArgumentException being raised suggests that Elasticsearch is trying to prevent non-cacheable queries in certain contexts. When you're working with real-time features like date math with the keyword now, the query becomes non-cacheable since the now value changes over time.

Instead of using now-6M in the query directly, you can compute the actual date value in your application and then use that specific date in the query. This way, you avoid the dynamic aspect of now, making the query cacheable. (For example, if today's date is 2023-09-19, the date 6 months ago would be 2023-03-19, and you can use this date directly in your query.)