Getting Parse error for elasticsearch-py

Question

I am trying to search my entire elasticsearch data for a certain word "tsbu" within a time range. When I try running this, I get a SearchParseException and Parse Failure.

es = Elasticsearch()

doc = {
        "query": {
            "match" : { 
                "message" : "tsbu"
            }
        }, 
        "range" : { 
            "@timestamp" : { 
                "gte" : "2015-06-09T14:44:00.000Z", 
                "lte" : "2015-06-09T14:50:00.000Z"
            }
        }
    }

print es.search(index="_all", body=doc)

The complete error I get is:

Traceback (most recent call last):
  File "essearch.py", line 22, in <module>
print es.search(index="_all", body=doc)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 504, in search
params=params, body=body)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 89, in perform_request
self._raise_error(response.status, raw_data)
  File "/usr/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[mPhuId4qSpa5osrqfeG5Tw][.kibana][0]: SearchParseException[[.kibana][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[.kibana][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][0]: SearchParseException[[logstash-2015.06.08][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][0]: SearchParseException[[logstash-2015.06.09][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][0]: SearchParseException[[logstash-2015.06.10][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][0]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][1]: SearchParseException[[logstash-2015.06.08][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][1]: SearchParseException[[logstash-2015.06.09][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][1]: SearchParseException[[logstash-2015.06.10][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][1]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][2]: SearchParseException[[logstash-2015.06.08][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][2]: SearchParseException[[logstash-2015.06.09][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][2]: SearchParseException[[logstash-2015.06.10][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][2]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][3]: SearchParseException[[logstash-2015.06.08][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][3]: SearchParseException[[logstash-2015.06.09][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][3]: SearchParseException[[logstash-2015.06.10][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][3]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.08][4]: SearchParseException[[logstash-2015.06.08][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.08][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.09][4]: SearchParseException[[logstash-2015.06.09][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.09][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }{[mPhuId4qSpa5osrqfeG5Tw][logstash-2015.06.10][4]: SearchParseException[[logstash-2015.06.10][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [Failed to parse source [{"query": {"match": {"message": "tsbu"}}, "range": {"@timestamp": {"gte": "2015-06-09T14:44:00.000Z", "lte": "2015-06-09T14:50:00.000Z"}}}]]]; nested: SearchParseException[[logstash-2015.06.10][4]: query[message:tsbu],from[-1],size[-1]: Parse Failure [No parser for element [range]]]; }]')

Answer 1

Your query is almost correct. The error your get states ...Parse Failure [Failed to parse source..., which basically means that your query is ill-formed and doesn't comply to the Query DSL.

The range query needs to be combined with the match query (using a bool/must query) and both need to be nested inside the query, like this

doc = {
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "message": "tsbu"
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2015-06-09T14:44:00.000Z",
              "lte": "2015-06-09T14:50:00.000Z"
            }
          }
        }
      ]
    }
  }
}

The end result is that you need tsbu to match the message field AND the @timestamp to be within the specified date range.

UPDATE

In order to have either message match tsbu or host match vcs but still require the timestamp to match, you can do it like this:

doc = {
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "message": "tsbu"
          }
        },
        {
          "match": {
            "host": "vcs"
          }
        }
      ],
      "must": [
        {
          "range": {
            "@timestamp": {
              "gte": "2015-06-09T14:44:00.000Z",
              "lte": "2015-06-09T14:50:00.000Z"
            }
          }
        }
      ]
    }
  }
}