Update elastic search nested field based on query

Question

I have an Elasticsearch index named pollstat with mapping as follows:

{
  "pollstat" : {
    "mappings" : {
      "dynamic" : "false",
      "properties" : {
        "dt" : {
          "properties" : {
            "dte" : {
              "type" : "date"
            },
            "is_polled" : {
              "type" : "boolean"
            }
          }
        },
        "is_profiled" : {
          "type" : "boolean"
        },
        "maid" : {
          "type" : "keyword"
        }
      }
    }
  }
}

The above index is created using:

curl -XPUT "http://localhost:9200/pollstat" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "maid" : {
        "type" : "keyword"
      },
      "dt" : {
        "type" : "object",
    "properties": {
        "dte" : {"type":"date"},
        "is_polled" : {"type":"boolean"}
    }
      },
      "is_profiled" : {
        "type" : "boolean"
      }
    },
  "dynamic":false
  }

}'

To add data into this index, I am using the following code:

curl -X POST "localhost:9200/pollstat/_doc/?pretty" -H 'Content-Type: application/json' -d'{"maid" : "fans", "dt" : [{"dte": "2022-03-19", "is_polled":true } ], "is_profiled":true } '

This is working.

The requirement is to append the dt field when a particular maid polls data on a specific date. In this case, if the maid fans polls data for another day, I want to append the same to the dt field.

I used the following code, which takes the document id to update the document.

curl -X POST "localhost:9200/pollstat/_doc/hQh4oH8BPfXX63hBUbPN/_update?pretty" -H 'Content-Type: application/json' -d'{"script": {"source": "ctx._source.dt.addAll(params.dt)", "params": {"dt": [{ "dte": "2019-07-16", "is_polled": true }, { "dte": "2019-07-17", "is_polled": false } ] } } } '

This is also working

However, my application does not have visibility to the document id but gets the maid. The maid is also as unique as the document id. Hence to update a specific maid, I was trying to do the same with a query on maid.

I used the following code:

curl -X POST "localhost:9200/pollstat/_update_by_query?pretty" -H 'Content-Type: application/json' -d'"query": {"match": { "maid": "fans" }, "script": {"source": "ctx._source.dt.addAll(params.dt)", "params": {"dt": [{ "dte": "2019-07-18", "is_polled": true }, { "dte": "2019-07-19", "is_polled": false } ] } } }'

This code executes without an error and I am getting the following update status as well:

{
  "took" : 8,
  "timed_out" : false,
  "total" : 1,
  "updated" : 1,
  "deleted" : 0,
  "batches" : 1,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [ ]
}

However my index is not getting updated.

Answer 1

Since the maid field has type keyword, I had to use the query->term instead of query->match. The final query is as follows:

curl -X POST "localhost:9200/pollstat/_update_by_query?pretty" -H 'Content-Type: application/json' -d'
{
    "query": {    
         "term": { "maid": "fans" }},
        "script": {
        "source": "ctx._source.dt.addAll(params.dt)",
        "params": {
            "dt": [
                { "dte": "2019-07-18", "is_polled": true },
                { "dte": "2019-07-19", "is_polled": false }
            ]
        }
    }
}
'

Posting this answer for others reference.