[alisonshao]

Problem

Reverts #31369, which introduced a deterministic correctness regression on main: the Qwen3-Next decode-cache-hit logprobs diverge from a cold recompute, failing the nightly 4-gpu-h100 test since 2026-07-17:

test/registered/models_e2e/test_qwen3_next_models.py::test_input_output_logprobs_match_decode_cache_hit_helper
AssertionError: avg_kl_div=0.006489 > threshold=0.002 for Qwen/Qwen3-Next-80B-A3B-Instruct

(identical kl_div on every retry → deterministic, not flaky).

Bisect

Pinned to #31369 (0675d3033f):

• kl_div was ≤0.0005 (pass) at dc0b3eb and jumped to 0.0065 (fail) at 681c2235; Fix mamba track-boundary seqlen under overlap scheduler #31369 is absent in the good run, present in the bad run.
• It is the only commit in that window touching _mamba_check_track_boundary / batch_result_processor.py / the mamba track-boundary logic.
• It rewrites the exact function that records the mamba tracking boundary for cache-hit decode — which is precisely what the failing test validates.

Why it regressed

#31369 changed the boundary seqlen from req.kv_committed_len to len(req.origin_input_ids) + len(req.output_ids) - 1. That new seqlen mislabels the mamba track boundary for the decode-cache-hit case, so a reused/donated tracked prefix extends from a state a cold prefill recompute would not produce — exactly the mismatch the test catches (and the symptom #31369's own description warns about).

Scope

Reverting restores the previously-passing behavior. The original overlap-scheduler boundary issue #31369 aimed at should re-land through the open #29792 ("Fix Mamba track-boundary bookkeeping under overlap scheduling"), which targets the same code with the correct bookkeeping.

Hardware A/B confirmation (revert → kl_div back under threshold) on a 4×H100 devbox is in progress; will update this PR with the before/after numbers.

---

CI States

Latest PR Test (Base): ❌ Run #29619940928
Latest PR Test (Extra): ❌ Run #29619940825

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[alisonshao]

/rerun-test test_qwen3_next_models.py

[github-actions]

Results for /rerun-test test_qwen3_next_models.py:

🚀 4-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_qwen3_next_models.py

[alisonshao]

/rerun-test test/registered/models_e2e/test_qwen3_next_models.py

[github-actions]

Results for /rerun-test test/registered/models_e2e/test_qwen3_next_models.py:

🚀 4-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_qwen3_next_models.py

[alisonshao]

Hardware-confirmed the A/B on 4-gpu-h100 (CI):

• Bad commit 681c2235 (nightly): fails deterministically — avg_kl_div 0.0037–0.0065 > 0.002.
• This revert: /rerun-test test_qwen3_next_models.py → passed on 4-gpu-h100 (run).

So #31369 is confirmed as the regressor and this revert restores the passing behavior. The original overlap-scheduler boundary fix should re-land via #29792.