Repository navigation
Add cluster_id to the mysql.tidb #59476
Description
Feature Request
Describe the feature you'd like:
Sometimes the external software will need to know the CLUSTER_ID to know whether the two TiDB instances are from the same cluster. For example: the CDC can use it to decide whether the upstream and downstream are from the same cluster and give an error to the user.
The CLUSTER_ID is already generated and stored in PD, but it's not exposed from TiDB side.
Describe alternatives you've considered:
I've considered two workaround before having this feature implemented:
Workaround 1: Use CLUSTER_INFO.SERVER_ID
Currently, we have a workaround to use CLUSTER_INFO.SERVER_ID to decide whether two servers are from the same cluster. For example, with a given server_id from the SERVER1, you can query SELECT count(1) FROM information_schema.cluster_info WHERE server_id = ? in the SERVER2 to know whether the two servers are from the same cluster.
However, a cluster may have a lot of servers and the possibility of conflict server id is not small enough. For example, if I have two cluster and each one of them has 100 server, the possibility of having conflict server id is:
When I have 200 servers, the possibility is:
Workaround 2: Sort and Hash CLUSTER_INFO.SERVER_ID
As using one SERVER_ID may face conflict, we can use the information provided by all servers, and hash them together. Run SELECT SERVER_ID FROM information_schema.cluster_info order by SERVER_ID and hash them. Now it should generate an unique id for each cluster. However, this ID will change during the scaling of the cluster, so it may have some bugs when we are generating the hash during scaling.
The both two workarounds are not that elegant.
Teachability, Documentation, Adoption, Migration Strategy:
- Add a new row to the
mysql.tidb:CLUSTER_ID xxxxx
Activity
YangKeao commentedon Feb 12, 2025
The only concern is that the
CLUSTER_IDfor all servers should be the same. It'll be a little strange 🤦 .Defined2014 commentedon Feb 12, 2025
How about add it into
mysql.tidbtable?YangKeao commentedon Feb 12, 2025
Good @Defined2014 . I think it's better. The only problem is that it's a real table, so we'll need to update the information during
bootstrap/upgraderight? (Assume that thecluster_idof a cluster will never change).Should we also sync the cluster_id regularly (or for each restart) after bootstrap?
cc @bb7133 and @wlwilliamx
Defined2014 commentedon Feb 12, 2025
I think you can update it anytime. Because it also contains
tikv_gc_last_run_timewhich will be updated when gc is triggered. But I am not sure whether thecluster_idvalue will change. Intuitively, it should not change.wlwilliamx commentedon Feb 12, 2025
I think adding the
cluster_idcolumn to theCLUSTER_INFOtable would be better. First, intuitively, if users need to know thecluster_idinformation, they would naturally look for it in theCLUSTER_INFOtable. Additionally, since theCLUSTER_INFOtable already records information about all nodes, users can easily find thecluster_idof a specific node—for example, if they want to check thecluster_idof a TiKV node. If thecluster_idcolumn is placed in themysql.tidbtable, it would still not be convenient for all TiDB nodes to access this information. Adding it toCLUSTER_INFOensures that thecluster_idof any node can be easily identified.PS, recording
cluster_idinformation for all instances is not a bad idea, as it allows all components of TiDB (e.g., TiKV, PD, TiDB itself) to conveniently query thecluster_idof any instance. This centralized approach improves consistency and accessibility across the system.[-]Add `CLUSTER_ID` to the `information_schema.CLUSTER_INFO`[/-][+]Add `CLUSTER_ID` to the `mysql.tidb`[/+]YangKeao commentedon Feb 13, 2025
We've decided to insert new row
cluster_id xxxto themysql.tidbtable, instead of exposing it throughinformation_schema.cluster_info. I'll create a new PR for it.[-]Add `CLUSTER_ID` to the `mysql.tidb`[/-][+]Add `cluster_id` to the `mysql.tidb`[/+]bootstrap: add `cluster_id` to the `mysql.tidb` table (pingcap#59511)
bootstrap: add `cluster_id` to the `mysql.tidb` table (pingcap#59511)
bootstrap: add `cluster_id` to the `mysql.tidb` table (#59511) (#65564)