I would like to designate certain tables as replicated to all TiKV stores such that they are always available to join with locally (thereby reducing expensive distributed joins at the TiDB level). This would allow the TiKV coprocessor to join locally to this table because it's always available (ie: replicated to every TiKV). In the OLAP terminology of "dimensions" and "facts", this is a dimension table. In this scenario, I'd like to shard facts and replicate dimensions. It appears that TiDB treats everything as a sharded fact. Can this be done? If not, can it be approximated with some other technique? How amenable is the code base to allowing this type of feature?
Related Questions in DISTRIBUTED-DATABASE
- What's the best practice to use OceanBase in cloud environment
- Load pre-training parameters trained on a single GPU on multi GPUS on a single machine
- Optimize database for top-k range query
- Bigtable secondary indexes : Best practices/Recommended-ways
- Mongodb transaction if document has multiple copies
- What happens to long running clickhouse updates if the client dies?
- Legal Hierarchical Quorums in Zookeeper
- How to handle data migrations in distributed microservice databases
- The relationship between Paxos family and data consistency
- How to remove all records from a DFS table but keep its schema?
- How to perform a table join on two DFS tables and update the left table based on its result?
- What is meant by Distributed System?
- Add on-premise CockroachDB node to a cluster hosted in Kubernetes
- Creating and migrating a devise-driven User model in a main/replica context
- Strong consistency and replication factor
Related Questions in TIDB
- Is it safe to restart servers for maintenance in tikv cluster?
- Configuring the node-exporter in a TiDB cluster deployment via TiUP
- Deleting data from MySQL table periodically
- In TiDB is it possible to change the SHARD_ROW_ID_BITS later on after inserting some data?
- Does TiDB Serverless support TiFlash for analytical jobs (OLAP) just like TiDB?
- Which path should I put config.toml in TiDB docker container?
- How to work around TiCDC atomic transactional limitations?
- Error P1001: Unable to reach database server when following Prisma's Start from Scratch guide for relational databases with Node.js and MySQL
- How does the tide-operator start a pd cluster?
- Does TiDB's range based sharding nature have actual read workload benefits?
- In TiDB how do you work around the issue where even for indexes, you can't write continuous values to avoid hotspots?
- In TiDB, how to use snowflake style uuid without causing hot spot by doing bit reverse?
- For TiDB, does incremental value in secondary index cause hotspots too?
- How to support "FULL JOIN" clause
- How to drop a table, the name with â\tâ in tidb?
Related Questions in TIKV
- Is it safe to restart servers for maintenance in tikv cluster?
- Increased replication in TiKV cluster, but storage use did not increase after applying
- TiDB executing full table scan
- TiKV is sufficient for both OLTP and OLAP workloads?
- specifying the TiKV cluster endpoint as the backing data store error
- How can I set memory capacity of each TIKV?
- What different betweent optimistic and pessimistic in tikv?
- what dose lockkeys in tikv api use for?
- Key Value Database Modeling for searchability
- Range query using TiKV database
- How is data replicated among 2 different TiKv clusters?
- how to optimize count(*)queries on tidb
- Java-grpc and tikv-java: NoSuchFieldError: CONTEXT_SPAN_KEY
- TIKV java client on MacOS: Failed to init client for PD cluster
- What is "mult-raft" in TiKV?
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
At present, TiDB splits each table into regions, and do the replication in the region level. It's hard to replicate a table into each TiKV server, even if it only contains one region. For example there are 100 nodes in the TiKV cluster but the configured number of region replica is 5.
We don't need to do the join operation in the TiKV coprocessor. We can read each dimension table from TiKV to multiply TiDB nodes and associate each involved TiDB node a portion of the fact table according to the data distribution of the fact table. Thus the join operation is done in the TiDB layer.
The technique described in the above is not implemented yet. But it's already on our roadmap.