[vuuihc]

Motivation

Fixes #30233.

In PD disaggregation, scheduler validation can reject a request after decode has already allocated KV for the original prompt. Req.set_finish_with_abort() replaces origin_input_ids with a one-token placeholder, but the rejected request previously still entered the normal PD queues.

That can make prefill transfer a one-token handoff while decode waits for or decodes from KV allocated for the original prompt. The client may receive garbage output, a transfer timeout, or a generic 500 instead of the original 400.

Modifications

Stop rejected requests before PD queue admission

• PREFILL and DECODE modes finish already-aborted requests before normal bootstrap/preallocation queues.
• PREFILL creates only the sender/control-plane state needed to notify decode; it does not enqueue the request or transfer prompt KV.
• The immediate response preserves the original FINISH_ABORT.

Keep failed-room ownership explicit

• Mooncake tracks pre-queue terminal rooms separately from scheduler-owned rooms.
• Late metadata and periodic orphan cleanup only eagerly clear pre-queue terminal rooms.
• Metadata registration, sender abort, notification, and cleanup share the room-state lock.
• Per-endpoint notification failures are contained; successful destinations are remembered and only failed destinations are retried.

Preserve structured failures across decode ranks

• Terminal status messages carry the original failure reason and optional HTTP status while accepting legacy messages without those fields.
• Decode gathers (reason, propagation flag, status) across TP ranks after failure consensus and selects the available structured error before streaming the abort.
• Untyped transport failures remain HTTP 500.

Tests

Regression coverage includes:

• PREFILL and DECODE pre-queue abort bypass.
• Scheduler-owned rooms surviving late metadata and periodic cleanup.
• Idle orphan cleanup for pre-queue failures.
• Per-endpoint notification failure containment and retry.
• Failure wire-format round trips and legacy compatibility.
• Stale HTTP status removal.
• Cross-rank selection of the original HTTP 400 payload.

Local validation on current main:

Python 3.11 py_compile: passed
pre-commit AST/isort/ruff/format/codespell/registry checks: passed
deterministic source-level concurrency/failure harness: passed

The full SGLang unit modules were not runnable on this macOS host because the current dependency set requires compressed-tensors>=0.18 with Torch >=2.10. GitHub CI is the authoritative full-suite run.

---

CI States

Latest PR Test (Base): ❌ Run #34922492172
Latest PR Test (Extra): ❌ Run #34922492041
Latest PR Test (AMD ROCm 10): ❌ Run #34922492136

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[vuuihc]

@xiezhq-hermann @hnyls2002 this PR is mergeable and fixes the aborted-prefill / decode-transfer correctness issue from #30233. The focused unit coverage and local formatting/static checks are documented in the PR body.

Could you please help with review and add run-ci when convenient? Thanks!

[xiezhq-hermann]

@stmatengss @ShangmingCai can you help take a look?

[stmatengss]

@stmatengss @ShangmingCai can you help take a look?

Sure.

[Copilot]

🟡 Changes recommended

Failed-room cleanup has concurrency and idle-cleanup defects that can leak state or terminate the Mooncake bootstrap thread.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prevents aborted PD requests from entering KV queues and propagates Mooncake failure details to decode.

Changes:

• Short-circuits aborted prefill/decode requests before queue admission.
• Extends Mooncake failure messages with reasons and HTTP statuses.
• Adds regression coverage for abort propagation and cleanup.

File summaries

File	Description
scheduler.py	Handles aborted PD requests before queueing.
common/conn.py	Adds HTTP status tracking to transfer failures.
mooncake/conn.py	Propagates failures and manages failed rooms.
decode.py	Preserves propagated failure details.
test_disaggregation_wire.py	Tests wire format and room lifecycle.
test_priority_scheduling_disaggregation.py	Tests queue bypass and decode errors.

Review details

Suppressed comments (1)

python/sglang/srt/disaggregation/mooncake/conn.py:2272

• The scheduler thread can call try_notify_decode_failure_and_clear() concurrently with this bootstrap thread. If it clears the room after the final transfer_infos[room] assignment but before the subsequent length/status checks, this code can raise KeyError and kill the bootstrap thread; other interleavings can resurrect a cleared room as WaitingForInput. Make metadata registration, readiness testing, notification, and room clearing atomic under a shared room-state lock.

                    self.required_dst_info_num_table[room] = required_dst_info_num
                    if room not in self.transfer_infos:
                        self.transfer_infos[room] = {}

                    self.transfer_infos[room][mooncake_session_id] = (
                        TransferInfo.from_zmq(waiting_req_bytes)

• Files reviewed: 6/6 changed files
• Comments generated: 2
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[stmatengss]

Reviewed the current head, focusing on PD abort propagation and room ownership. The three inline issues below were reproduced with a small CPU harness that executes the relevant methods extracted from this commit, with transport/distributed state mocked. Full unit tests could not run locally because PyTorch is unavailable; no multi-node GPU E2E validation was performed.

[vuuihc]

@stmatengss I rebased this PR onto current main and addressed all three findings in 5bdb7f1365; details and regression coverage are in the inline replies. The PR is mergeable. Lint passes; the latest platform workflows were skipped by the 120-minute CI cooldown after the follow-up push. Could you please re-review when convenient?