I have an elasticsearch index which has multiple documents, now I want to update the index with some new documents which might also contain duplicates of the existing documents. What's the best way to do this? I'm using elasticsearch py for all CRUD operations
How to upsert documents in an existing elasticsearch index?
8.2k views Asked by David At
1
There are 1 answers
Related Questions in ELASTICSEARCH
- How does Elasticsearch do attribute filtering during knn (vector-based) retrieval?
- Elastic python to extract last 1hr tracing
- Elastic search not giving result when Hyphen is used in search text
- FluentD / Fluent-Bit: Concatenate multiple lines of log files and generate one JSON record for all key-value from each line
- Elasticsearch functional_score with parameter of type string array as input not working
- Elasticsearch - cascading http inputs from Airflow API
- AWS Opensearch - Restore snapshot - Failed to parse object: unknown field [uuid] found
- cluster block exception for system index of kibana
- What settings are best for elasticsearch query to find full word and half word
- OpenSearch - Bulk inserting Million rows from Pandas dataframe
- unable access to kibana
- PySpark elastic load fail with error SparkContext is stopping with exitCode 0
- How to use query combined to KNN with ElasticSearch?
- Facing logstash compatibility issues
- If the same document is ingested at two different times, how to have the same id in Elasticsearch
Related Questions in ELASTICSEARCH-PY
- Elasticsearch python client connection timed out
- How to validate a document against Elasticsearch mapping without indexing it in Python?
- elasticsearch-py HTTPS_PROXY Variable Ignored - Corporate Proxy
- About the `Elasticsearch.index()` method, is it using PUT or POST?
- Partial text-formatted date match with Elasticsearch regexp
- Get the phrase found by elastic search in a complex bool query
- Elasticsearch Python - No viable nodes were discovered on the initial sniff attempt
- Elasticsearch-Py Bulk Helper expected field [create], [delete], [index] or [update] but found [_op_type]'
- Elasticsearch autocomplete query
- Bulk indexing in elasticsearch 8.x : 'Action/metadata line [1] contains an unknown parameter [_type]'
- Provide the "script" argument in elasticsearch-py
- How can I bulk upload JSON records to AWS OpenSearch index using a python client library?
- Remote ElasticSearch cluster azure cloud connection using Python Client using http
- Sequential Searching Across Multiple Indexes In Elasticsearch
- Bulk index / create documents with elasticsearch for python
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Every update in elasticsearch deletes the old document and create a new document as the smallest unit of document collection is called segments in elastic-search which are immutable, hence when you index a new document or update any exiting documents, it gets into the new segments which are merged into bigger segments during the merge process.
Now even if you have duplicate data but with the same id, it will replace the existing document, and its fine and more performant than first fetching the document and than comparing both the documents to see if they are duplicate and than discard the update/upsert request from an application, rather than just index whatever if coming and ES will again insert the duplicate docs.