Delete records of a certain type from logstash/elasticsearch

Question

I'm about to embark upon importing a large number of records into elasticsearch (via logstash).

I'm sure I will make a few mistakes. As such, I would like to be able to easily delete the imported records from elasticsearch.

For now, I can just delete the indicies containing the imports. That only works because I won't have any other data on those same days.

In the future, I'd really like to query elasticsearch for all records of a certain type, and delete them.

In this case, the record type will be "nbu_job"

Answer 1

You can use the delete by query API to achieve that.

For instance, the following command will delete all documents of type nbu_job in the index your_index:

curl -XDELETE 'http://localhost:9200/your_index/_query?q=_type:nbu_job'

If you need to verify what is going to be deleted with the above command, I suggest you run the query first so you can see what is matched with this command (note that _query endpoint becomes _search !)

curl -XGET 'http://localhost:9200/your_index/_search?q=_type:nbu_job'

Note that if you do this, the mapping for nbu_job will still exist. So if you notice you've made errors in your mapping, you probably want to remove the nbu_job type altogether using

curl -XDELETE 'http://localhost:9200/your_index/nbu_job'

Of course, your_index is whatever index name you need to work on, it can also be an alias, a comma-separated list of indices (e.g. logstash-2015.06.11,logstash-2015.06.12), an index wildcard (e.g. logstash-*)