[ispobock]

Motivation

A prefix-cache hit on a hybrid-SWA model can decode off a conv state it never produced. The replayed prefill graph reads _capture_req_slots rows, but mamba_track_mask / mamba_track_indices keep whatever the previous replay left past the live batch, so the captured track scatter lands the current window in an earlier request's checkpoint. Whichever request later restores that prefix is wrong for its whole generation.

The sentinel tail directly above exists for this reason -- "the captured graph reads all req_slots entries, so stale values from the previous replay must be cleared" -- and clears six fields. The two track buffers live in the buffer registry instead, and their clear is guarded by bs != raw_bs in PrefillInputBuffers.fill_from; the prefill graph passes padded_bs == raw_bs because it pads tokens rather than requests, so that branch never runs.

This needs no speculative decoding. It takes the prefill CUDA graph, chunked prefill, concurrency and a prefix hit together, and disabling the prefill graph alone makes it go away. #34043 fixed a neighbouring failure on the spec path -- the scatter being specialised out of the capture -- whereas here the checkpoint is written and scattered to a stale destination; it still reproduces with #34043 in.

Modifications

Zero the two track rows alongside the six the sentinel tail already covers.

#34168 test_inkling_small_nvfp4.py drops the --disable-prefill-cuda-graph it was carrying. That flag was steering the deterministic KL case around this bug, and that case is the regression guard for it, so it belongs back on the graph path.

The alternative is to make padded_bs report the captured slot count, which makes fill_from's clear live and stops its bs != raw_bs branch from being dead code. That is the more honest shape -- while the request dimension claims to be unpadded, the next captured kernel that reads req_slots rows repeats this -- but it changes behaviour under every phase sharing those buffers, so this PR takes the local fix. Happy to switch.

Accuracy

Shrunken Inkling checkpoint, tp=1, 32 concurrent prompts, prefill graph on, --enable-deterministic-inference. Differing logprobs against a flushed reference prefill:

              prefill_cache_hit                match
           512 tokens    8 tokens
before     1022/16384      16/256              0
after         0/16384       0/256              0

The two corrupted checkpoints were observed rather than inferred: instrumenting the slot at insert time shows exactly two whose conv state differs from the same run without the graph, and their claimed track seqlens are the prefixes of the two requests that go wrong.

The A/B above is the shrunken checkpoint at tp=1. The CI case runs the real NVFP4 weights at tp=4, where the batch packs differently and picks different victims, so I have not reproduced-and-fixed on that exact configuration -- if it stays red, the same shape of gap exists somewhere else and the instrumentation above finds it.

---

CI States

Latest PR Test (Base): 🚫 Run #31327464791
Latest PR Test (Extra): ❌ Run #31327464735

[ispobock]

/rerun-test test/registered/models_e2e/test_inkling_small_nvfp4.py

[github-actions]

Results for /rerun-test test/registered/models_e2e/test_inkling_small_nvfp4.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_inkling_small_nvfp4.py

[ispobock]

/rerun-failed-ci

[ispobock]

https://github.com/sgl-project/sglang/actions/runs/31325697470/job/93275590138#step:7:1545