I'm writing 2.5 million records into Cassandra using a python program. The program finishes quickly but on querying the data, the records are reflected after a long time. The number of records gradually increase and it seems like the database is performing the writes to the tables in a queue fashion. The writes continue on till all the records are finished. Why do writes reflect late?
Why do writes through Cassandra Python Driver add records with a delay?
84 views Asked by Pradyumna Khawas At
1
There are 1 answers
Related Questions in PYTHON
- How to store a date/time in sqlite (or something similar to a date)
- Instagrapi recently showing HTTPError and UnknownError
- How to Retrieve Data from an MySQL Database and Display it in a GUI?
- How to create a regular expression to partition a string that terminates in either ": 45" or ",", without the ": "
- Python Geopandas unable to convert latitude longitude to points
- Influence of Unused FFN on Model Accuracy in PyTorch
- Seeking Python Libraries for Removing Extraneous Characters and Spaces in Text
- Writes to child subprocess.Popen.stdin don't work from within process group?
- Conda has two different python binarys (python and python3) with the same version for a single environment. Why?
- Problem with add new attribute in table with BOTO3 on python
- Can't install packages in python conda environment
- Setting diagonal of a matrix to zero
- List of numbers converted to list of strings to iterate over it. But receiving TypeError messages
- Basic Python Question: Shortening If Statements
- Python and regex, can't understand why some words are left out of the match
Related Questions in CASSANDRA
- how to create a chess board with Queen in the central position and all its moves in assembler code
- Passing arguments to ENTRYPOINT causes the container to start and run indefinitely
- Apache Cassandra Node Driver Connection
- Simulate Cassandra DB timeout
- How to update Cassandra Lucene index with a new column? rebuild or update index?
- Cassandra JDBC connection string for logstash
- Cassandra OversizedMessageException
- dsbulk unload is failing after ran couple of hours with OOM issue
- Cassandra: "Model keyspace not set" and "Connection name doesn't exist in the registry" Errors
- Unable to cqlsh to a cassandra docker container remotely
- Forward pagination with object mapper in java asyn
- Allow filter in cassandra query
- How to fix bytes unrepaired in cassandra
- Can't install Cassandra using RPM packages for RHEL 9
- Why can't get a connection to Cassandra running on Docker from a Spring Boot instace using spring-boot-starter-data-cassandra on first boot?
Related Questions in CASSANDRA-PYTHON-DRIVER
- Batch insert fails for table with a CQL map and nested UDT
- Can't connect to Astra DB with Python driver
- Error accessing Scylladb cluster outside docker container
- Cassandra Data Read Speed Slows Down
- Why cassandra python query result replaces column name's space with underscore?
- ConnectionRefusedError No connection could be made because the target machine actively refused it cassandra
- Python Cassandra driver: connect to Docker container on server - cassandra.UnresolvableContactPoints: {}
- CassandraDB and Python: using the UnixSocketEndPoint?
- cassandra-driver from DataStax: How to create Cassandra DB table from a Model?
- Why do writes through Cassandra Python Driver add records with a delay?
- Cassandra Python executing multiple statement from string using python drive?
- How to find the first host contacted by cassandra driver during connection?
- Optimize inserting data to Cassandra database through Python driver
- Cassandra NoHostAvailable mistake
- Cassandra cluster connect fails each 2nd time with NoHostAvailable
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Popular Tags
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
It is customary to provide a minimal code example plus steps to replicate the issue but you haven't provided much information.
My guess is that you've issued a lot of asynchronous writes which means that those queries get queued up because that's how asynchronous programming works. Until they eventually reach the cluster and get processed, you won't be able to immediately see the results.
In addition, you haven't provided information on how you're verifying the data so I'm going to make another guess and say you're doing a
SELECT COUNT(*)which requires a full table scan in Cassandra. Given that you've issued millions of writes, chances are the nodes are overloaded and take a while to respond.For what it's worth, if you are doing a
COUNT()you might be interested in this post where I've explained why it's bad to do it in Cassandra -- https://community.datastax.com/questions/6897/. Cheers!