Repository navigation
Init Stats May Be Very Slow When Using Information Schema V1 with Partitioned Tables #64509
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
select /*+ ORDER_INDEX(mysql.stats_histograms,tbl) */ HIGH_PRIORITY table_id, is_index, hist_id, distinct_count, version, null_count, cm_sketch, tot_col_size, stats_ver, correlation, flag, last_analyze_pos from mysql.stats_histograms where table_id >= 2192000 and table_id < 2192500 order by table_id;
| id | estRows | estCost | actRows | task | access object | execution info | operator info | memory | disk |
| Projection_8 | 250.00 | 492478.33 | 1670 | root | | time:7.62ms, loops:3, Concurrency:OFF | mysql.stats_histograms.table_id, mysql.stats_histograms.is_index, mysql.stats_histograms.hist_id, mysql.stats_histograms.distinct_count, mysql.stats_histograms.version, mysql.stats_histograms.null_count, mysql.stats_histograms.cm_sketch, mysql.stats_histograms.tot_col_size, mysql.stats_histograms.stats_ver, mysql.stats_histograms.correlation, mysql.stats_histograms.flag, mysql.stats_histograms.last_analyze_pos | 186.8 KB | N/A |
| └─Projection_12 | 250.00 | 492178.93 | 1670 | root | | time:7.58ms, loops:3, Concurrency:OFF | mysql.stats_histograms.table_id, mysql.stats_histograms.is_index, mysql.stats_histograms.hist_id, mysql.stats_histograms.distinct_count, mysql.stats_histograms.null_count, mysql.stats_histograms.tot_col_size, mysql.stats_histograms.version, mysql.stats_histograms.cm_sketch, mysql.stats_histograms.stats_ver, mysql.stats_histograms.flag, mysql.stats_histograms.correlation, mysql.stats_histograms.last_analyze_pos | 196.3 KB | N/A |
| └─IndexLookUp_11 | 250.00 | 491879.53 | 1670 | root | | time:7.56ms, loops:3, index_task: {total_time: 2.08ms, fetch_handle: 1.91ms, build: 153.8µs, wait: 20.4µs}, table_task: {total_time: 8.17ms, num: 2, concurrency: 5}, next: {wait_index: 2.09ms, wait_table_lookup_build: 252.4µs, wait_table_lookup_resp: 4.89ms} | | 374.0 KB | N/A |
| ├─IndexRangeScan_9(Build) | 250.00 | 61050.00 | 1670 | cop[tikv] | table:stats_histograms, index:tbl(table_id, is_index, hist_id) | time:1.86ms, loops:4, cop_task: {num: 1, max: 1.8ms, proc_keys: 1670, tot_proc: 638.1µs, tot_wait: 17.3µs, copr_cache_hit_ratio: 0.00, build_task_duration: 10.1µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.79ms}}, tikv_task:{time:0s, loops:6}, scan_detail: {total_process_keys: 1670, total_process_keys_size: 103540, total_keys: 1671, get_snapshot_time: 5.26µs, rocksdb: {key_skipped_count: 1670, block: {cache_hit_count: 11}}}, time_detail: {total_process_time: 638.1µs, total_wait_time: 17.3µs, tikv_wall_time: 698.6µs} | range:[1742000,1742500), keep order:true, stats:pseudo | N/A | N/A |
| └─TableRowIDScan_10(Probe) | 250.00 | 104559.57 | 1670 | cop[tikv] | table:stats_histograms | time:7.54ms, loops:4, cop_task: {num: 7, max: 4.51ms, min: 740.1µs, avg: 1.91ms, p95: 4.51ms, max_proc_keys: 624, p95_proc_keys: 624, tot_proc: 8.94ms, tot_wait: 152.5µs, copr_cache_hit_ratio: 0.00, build_task_duration: 44.9µs, max_distsql_concurrency: 4}, rpc_info:{Cop:{num_rpc:7, total_time:13.3ms}}, tikv_task:{proc max:0s, min:0s, avg: 0s, p80:0s, p95:0s, iters:22, tasks:7}, scan_detail: {total_process_keys: 1670, total_process_keys_size: 148630, total_keys: 1912, get_snapshot_time: 56.8µs, rocksdb: {key_skipped_count: 1525, block: {cache_hit_count: 2554}}}, time_detail: {total_process_time: 8.94ms, total_suspend_time: 34.3µs, total_wait_time: 152.5µs, tikv_wall_time: 9.73ms} | keep order:false, stats:pseudo | N/A | N/A |2. What did you expect to see? (Required)
No slow queries.
3. What did you see instead (Required)
Very slow queries.
4. What is your TiDB version? (Required)
master, v8.5.x
Activity
0xPoe commentedon Nov 16, 2025
Introduced by: #57574
0xPoe commentedon Nov 16, 2025
The issue comes from:
tidb/pkg/infoschema/infoschema.go
Line 434 in 268e1df
The CPU is heavily consumed after the call to FindTableByPartitionID. After reviewing the entire implementation, I found that it is an O(n) function. This means each invocation may trigger a full scan of all 200,000 tables to retrieve partition table information.
In this issue, the number of full-table traversals grows linearly with the number of tables. The more partitioned tables, total tables, or columns there are, the more significant the problem becomes. For clusters with a large number of tables, the fallback of init stats becomes more noticeable. Since init stats affects the readiness time of each node during rolling restarts, the primary impact is that the overall rolling restart duration becomes longer — and the larger the number of tables, the longer it gets.
0xPoe commentedon Nov 16, 2025
cc @lance6716
kennedy8312 commentedon Nov 18, 2025
/type regression
statistics: cache partition IDs to global IDs map when getting the ta…
statistics: cache partition IDs to global IDs map when getting the ta…
statistics: cache partition IDs to global IDs map when getting the ta…