Repository navigation
import into merge stage is extremely slow for large datasets due to suboptimal resource usage #60375
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
- datesize 8T with about 48 billion rows
- DXF is enabled
- import into xxx from s3:xxx
2. What did you expect to see? (Required)
The merge phase should be optimized to make full use of all available TiDB resources
3. What did you see instead (Required)
Only 2 subtasks were allocated, and while one finishes the merge step quickly, the other takes nearly 2 hours.
[next-step=merge-sort] [subtask-count=2]
4. What is your TiDB version? (Required)
v8.5.1
Activity
disttask: distribute merge sort subtasks to all available nodes (#60395)
disttask: distribute merge sort subtasks to all available nodes (ping…
disttask: distribute merge sort subtasks to all available nodes (#60395…