Index policy or Index template for Elasticsearh

Question

I have elasticsearch cluster for storing logs, and i have indices like this

logs-2021.01.01
logs-2021.01.02
logs.2021.01.03 ...etc

so indices creates at daily basis, and i have index template for this indices

PUT _index_template/template_1
{
  "index_patterns": ["logs*"],
  "template": {
    "settings": {
      "number_of_shards": 6,
      "number_of_replicas": 1
    }

but I want to make sure that indexes that are older than 1 day have 0 replicas to save disk space, and indexes that are younger than 1 day remain with 1 replica (so that in case of server loss, I have data for today)

how can i do this using elasticsearch way? i think about bash script that executes by cron , which get all of the indices which older than 1 day and make 0 replica, but i don't want to use external scripts to do that Thank you for you help

Answer 1

You can use ILM (Index life cycle management) concept of the Elasticsearch.

I this, you can create policy with different state and perform some action in each state.

You can give the condition, when the index gets migrated to next state. you can give your condition base on your scenario.

PUT _ilm/policy/my_policy
{
  "policy": {
    "phases": {
      "warm": {
        "actions": {
          "allocate" : {
            "number_of_replicas" : 0
          }
        }
      }
    }
  }
}

https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-allocate.html

This is not the full proof policy but you can use this concept for your scenario.