Ignoring duplicates within elastic search

Question

I have many records where the msg is 'a'. Some of these records have the same type.

I'm trying to write a query that counts the number of with msg 'a', but doesn't count duplicates.

Example:

• 1: msg = 'a', type = 'b'
• 2: msg = 'a', type = 'b'
• 3: msg = 'a', type = 'c'

This should return a count of two because the first and second records have the same type and are only counted once.

Here is my query so far.

body: {
  query: {
    bool: {
      must: [
        {
          range: {
            "@timestamp" => { from: 'now-1d', to: 'now' }
          }
        },
        { match: { msg: 'a' }}
      ]
    }
  }
}

Any help is appreciated!

Answer 1

Try using aggregations they'll count it for you :) Read here: https://www.elastic.co/guide/en/elasticsearch/reference/5.5/search-aggregations-bucket-terms-aggregation.html

And try something like this:

body:{
query: {
    bool: {
      must: [
        {
          range: {
            "@timestamp" => { from: 'now-1d', to: 'now' }
          }
        },
        { match: { msg: 'a' }}
      ]
    }
  }
},
  aggs:{
     "type":{
        "terms":{
              "field":"type"
         }
     }
  }
}