I need to reindex data periodically like every 1 hour using a custom Query DSL.
I reindex a subset of documents, i use an ingest pipeline and a custom script to update the new index.
It looks like that (with changed values):
POST _reindex?wait_for_completion=false
{
"source": {
"index": "index",
"query": {
"bool": {
"filter": {
"terms": {
"Domain.keyword": ["Custom.Value0", "Custom.Value1", "Custom.Value2"]
}
}
}
}
},
"dest": {
"index": "new-index-clean",
"pipeline": "my-pipeline"
},
"script": {
"lang": "painless",
"source": "if (!ctx._source.containsKey(\"Field\") || ctx._source.Field == null) {ctx._source.Field = ctx._index}"
}
}
I installed Job Scheduler on my Opensearch instance and i saw that you can create custom plugins to run your jobs but i can't install custom plugins and i can't find documentation on how to use it otherwise.
I tried Rollups or Transorm job but i don't have the option i need for my reindex.