Opensearch - Reindex periodically using Query DSL

26 views Asked by At

I need to reindex data periodically like every 1 hour using a custom Query DSL.

I reindex a subset of documents, i use an ingest pipeline and a custom script to update the new index.

It looks like that (with changed values):

POST _reindex?wait_for_completion=false
{
  "source": {
    "index": "index",
    "query": {
      "bool": {
        "filter": {
            "terms": {
                "Domain.keyword":  ["Custom.Value0", "Custom.Value1", "Custom.Value2"]
            }
        }
      }
    }
  },
  "dest": {
    "index": "new-index-clean",
    "pipeline": "my-pipeline"
  },
  "script": {
      "lang": "painless",
      "source": "if (!ctx._source.containsKey(\"Field\") || ctx._source.Field == null) {ctx._source.Field = ctx._index}"
    }
}

I installed Job Scheduler on my Opensearch instance and i saw that you can create custom plugins to run your jobs but i can't install custom plugins and i can't find documentation on how to use it otherwise.

I tried Rollups or Transorm job but i don't have the option i need for my reindex.

0

There are 0 answers