Elastic search, watcher access dotted field names in the result set

Question

I created a query for a elastic search watcher setup. The result set looks like this:

"_index": "transaction_broker-2017.09.15",
        "_type": "transaction_broker",
        "_id": "AV6Fn_UQ9KbnKce40avY",
        "_score": 3.8539968,
        "_source": {
          "tbroker.workitem.sync_check.tbroker_value": 7000,
          "source": "/logs/web/tomcat/tbroker.log.json",
          "type": "transaction_broker",
          "tbroker.job.instance_id": "lixporta-p00.xxxxxxx.15053054001381505305457198",
          "tbroker.workitem.sync_check.backend_total_value": 6995,
          "tbroker.appversion": "1.1.135-180",
          "@version": 1,
          "beat": {
            "hostname": "lixporta-p00",
            "name": "lixporta-p00",
            "version": "5.1.1"

In the action section, I can access the fields by using:

"actions": {
    "my-logging-action": {
      "logging": {
        "text": "There are {{ctx.payload.hits.hits.0._source.....

After the source tag, I use for example the "type" field from the list above. Other example is:

"ctx.payload.hits.hits.0._source.beat.hostname"

This works pretty fine... But it is not possible to use a field like

"tbroker.workitem.sync_check.tbroker_value"

The parser thinks that this fields are nested, but this is only a fieldname with dots in it. Is there any possiblity to "escape" this fieldname? Anyone who also have had this problem ?

Many thanks & best regards Claus

Answer 1

I think the following should work:

{{#ctx.payload.hits.hits.0._source}}{{tbroker.workitem.sync_check.tbroker_value}}{{/ctx.payload.hits.hits.0._source}}

It is a limitation of Mustache and this is a workaround.

Another example may help - when in a context looping through hits (I have added // comments purely for clarity - they aren't valid Mustache syntax & should be removed):

{{#ctx.payload.hits.hits}}
  // This works fine
  {{_source.foo}}
  // Not working if each hit's _source contains "bar.baz", not nested "bar">"baz"
  {{_source.bar.baz}}
{{/ctx.payload.hits.hits}}

Applying the same workaround by adding an extra context/section:

{{#ctx.payload.hits.hits}}
  // Put us in the context of [the current hit] > _source
  {{#_source}}
    // Now both of these work...
    {{foo}}
    // ...including this one containing a dot (yay!)
    {{bar.baz}}
  {{/_source}}
{{/ctx.payload.hits.hits}}

Answer 2

There is no way to directly access source fields that have dots in them, but if you apply a transform like this:

"transform": {
  "script": {
    "inline": "return [ 'host' : ctx.payload.hits.hits[0]._source.host, 'tbroker_value' : ctx.payload.hits.hits[0]._source['tbroker.workitem.sync_check.tbroker_value']]",
   "lang": "painless"
  }
}

and then you can use {{ctx.payload.host}} and {{ctx.payload.tbroker_value}} in your action.