Repository navigation
dead chanel in UpdateNewAndDoneWatch #61782
Copy link
Copy link
Labels
affects-8.1This bug affects the 8.1.x(LTS) versions.This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.This bug affects the 8.5.x(LTS) versions.affects-9.0This bug affects the 9.0.x versions.This bug affects the 9.0.x versions.component/pdimpact/upgradeseverity/majortype/bugThe issue is confirmed as a bug.The issue is confirmed as a bug.
Description
Bug Report
Please answer these questions before submitting your issue. Thanks!
1. Minimal reproduce step (Required)
It started here
Lines 75 to 80 in e601725
| err = do.runawayManager.UpdateNewAndDoneWatch() | |
| if err == nil { | |
| logutil.BgLogger().Info("preparations for the runaway manager are finished and start runaway manager loop") | |
| do.wg.Run(do.runawayManager.RunawayRecordFlushLoop, "runawayRecordFlushLoop") | |
| do.wg.Run(do.runawayManager.RunawayWatchSyncLoop, "runawayWatchSyncLoop") | |
| do.runawayManager.MarkSyncerInitialized() |
will insert record into channels here
tidb/pkg/resourcegroup/runaway/manager.go
Lines 424 to 426 in 6a12f31
| } else if ttl <= 0 { | |
| rm.staleQuarantineRecord <- record | |
| return |
The receive loop however is started later than the first call of
UpdateNewAndDoneWatch(RunawayRecordFlushLoop)
tidb/pkg/resourcegroup/runaway/manager.go
Line 211 in 6a12f31
| case r := <-staleQuarantineRecordCh: |
If the channel exceeds the maximum(1024), it blocks.
introduced by #52283
2. What did you expect to see? (Required)
non blocking.
3. What did you see instead (Required)
runawayStartLoop blocks forever
4. What is your TiDB version? (Required)
v8.1.0 and later
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
affects-8.1This bug affects the 8.1.x(LTS) versions.This bug affects the 8.1.x(LTS) versions.affects-8.5This bug affects the 8.5.x(LTS) versions.This bug affects the 8.5.x(LTS) versions.affects-9.0This bug affects the 9.0.x versions.This bug affects the 9.0.x versions.component/pdimpact/upgradeseverity/majortype/bugThe issue is confirmed as a bug.The issue is confirmed as a bug.
Activity
fix(runaway): resolve the dead channel in UpdateNewAndDoneWatch (#61795)
fix(runaway): resolve the dead channel in UpdateNewAndDoneWatch (#61795…
seiya-annie commentedon Aug 26, 2025
Upgrade branch meets this issue:
source: <=8.1.0
target: v8.1.1-v8.5.2
blocking stage: post upgrade
workaround: manually delete the rows in the table on the working tidb instance.