elascticsearch roll over based on ilm policy with keeping application/consumer as-is

Question

I have an elasticsearch service which is a logging backend for our applications. Application code is not open to us, so we can not change the application. applications are calling elasticsearch in certain points in flow via http as below.

POST order-service-2022.04.23/_doc
{
  "message": "order created",
  "@timestamp": "1591890613"
}

in the later day they are calling the same service like below. only date is changing.

POST order-service-2022.04.24/_doc
{
  "message": "order created",
  "@timestamp": "1591890613"
}

The problem is we can not set up an ILM policy with out changing the client behaviour. The aim is roll over the index based on some criteria, since we can not change the application code, http call should be like that in application side. what we have tried is :

index template :

PUT _index_template/order_template
{
  "index_patterns": [
    "order-*-*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 0,
      "index.lifecycle.name": "order_policy",
      "index.lifecycle.rollover_alias": "order"
    }
  }
}

ilm policy

PUT _ilm/policy/order_policy
{
  "policy": {
    "phases": {
      "hot": {                                
        "actions": {
          "rollover": {
            "max_primary_shard_size": "1GB", 
            "max_age": "1d",
            "max_docs": 500
          }
        }
      },
      "delete": {
        "min_age": "2d",                     
        "actions": {
          "delete": {}                        
        }
      }
    }
  }
}

seed index :

PUT order-service-2022.04.23
{
  "aliases": {
    "order": {
      "is_write_index": true
    }
  }
}

during doc insert to elastic successful result.

but during rollup for only one day give below exception

POST /order/_rollover
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "index name [order-service-2022.04.23] does not match pattern '^.*-\\d+$'"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "index name [order-service-2022.04.23] does not match pattern '^.*-\\d+$'"
  },
  "status" : 400
}

second issue how the up coming days need to rollup since no alias for them..

Answer 1

First, it looks like the application code is doing the "rollover" by itself since it's changing the index name everyday. So if you can't change the application code, you sort of already have some rollover logic but implemented directly in your application code.

The error you get is because the initial seed index MUST end with a sequential number otherwise it won't work.

The regular expression ^.*-\\d+$ means "any index name that ends with one or more digits"

                            add this
                               |
                               v
PUT order-service-2022.04.23-00001
{
  "aliases": {
    "order": {
      "is_write_index": true
    }
  }
}

This also means that the application code that makes up the index name would NOT be able to know what the current sequence number is, and hence, MUST write directly to the order write alias instead, otherwise it's not going to work.