[YAMY1234]

Motivation

DeepSeek-V4's C4 indexer currently evaluates logits directly from paged KV storage. For long-context, unpadded single-request EXTEND batches, gathering the C4 index keys and scales into contiguous storage enables the faster non-paged DeepGEMM fp8_mqa_logits path while preserving the existing top-k semantics.

This PR adds that path behind a default-off environment flag.

Related to #23602.

Modifications

• Add a default-off non-paged C4 indexer path for eligible single-request, unpadded EXTEND batches.
• Reuse one validated gather plan across indexer layers and fall back to the existing paged path for unsupported inputs, backends, or graph modes.
• Add focused CPU tests for eligibility, plan geometry, extreme-size fail-closed behavior, and the gather/dispatch contract.

Accuracy Tests

pip install git+https://github.com/sgl-project/sgl-eval

sgl-eval run gsm8k \
  --model deepseek-ai/DeepSeek-V4-Pro --api-key <api-key> \
  --n-repeats 1 --temperature 1.0 --top-p 0.95 --no-thinking \
  --out-dir /sgl-workspace/logs \
  --base-url http://localhost:30000/v1

sgl-eval run aime25 \
  --model deepseek-ai/DeepSeek-V4-Pro --api-key <api-key> \
  --n-repeats 16 --max-tokens 400000 \
  --temperature 1.0 --top-p 1.0 --thinking \
  --out-dir /sgl-workspace/logs \
  --base-url http://localhost:30000/v1

Evaluation	Result
GSM8K	96.59% (1,274 / 1,319; 0% error, 0% truncated)
AIME25 pass@1 (average of 16 repeats)	99.17% (476 / 480; pass@16 100%; majority@16 100%; 2 errors, 0 truncated)

Speed Tests and Profiling

End-to-end experiment

The treatment prefill worker used the following configuration; the baseline used the same command with SGLANG_OPT_DSV4_NONPAGED_INDEXER=0.

SGLANG_OPT_DSV4_NONPAGED_INDEXER=1 \
SGLANG_RADIX_FORCE_MISS=1 \
python3 -m sglang.launch_server \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --trust-remote-code \
  --disaggregation-mode prefill \
  --disaggregation-transfer-backend mooncake \
  --tp 4 --dp 4 --ep 4 \
  --enable-dp-attention --enable-dp-lm-head \
  --moe-a2a-backend deepep \
  --kv-cache-dtype fp8_e4m3 \
  --mem-fraction-static 0.9 \
  --max-running-requests 128 \
  --cuda-graph-max-bs 128 \
  --chunked-prefill-size 32768 \
  --context-length 262144 \
  --disable-flashinfer-autotune

Workload: fixed nominal-128K input, output length 1, concurrency 8, and server-side full-chunk prefill throughput over a fixed 300-second window.

Pair	Order / placement	Baseline median (tok/s/rank)	Non-paged median (tok/s/rank)	Pooled median gain	Rank-equal gain
1	baseline -> treatment	12,188.145	12,465.880	+2.279%	+2.210%
2	treatment -> baseline	11,940.050	12,555.300	+5.153%	+5.380%
3	same placement, baseline -> treatment	11,576.590	12,330.800	+6.515%	+6.451%
Final PR head eb08e8d517	same placement, baseline -> treatment	12,001.005	12,318.740	+2.648%	+2.708%

The equal-weight aggregate across the 3 prototype pairs x 4 DP ranks was +4.665%, with all 12/12 per-rank effects positive. The exact clean PR head was then rerun as a same-placement serial pair: rank-equal gain was +2.708%, pooled-median gain was +2.648%, and all 4/4 DP-rank effects were positive (+4.089% / +1.273% / +4.251% / +1.258%). Both arms used the same source, workload/configuration, and fixed 300-second analysis contract; the feature flag was the treatment difference. This single final-head pair validates direction but is not a stable-magnitude or default-on claim.

Kernel microbenchmark

Original context	Net indexer latency reduction, including gather
32K	19.1%
64K	26.6%
128K	23.0%

At 128K:

Paged logits	Gather	Non-paged logits	Gather + non-paged	Net change
2,870.2 us	104.0 us	2,105.1 us	2,209.1 us	-23.0%

Kernel Breakdown

Setup: 1× GB300 node (4 GPUs), TP4/DP4/EP4 with DP attention and FP8 KV cache. Fixed 128K ISL, OSL=1, concurrency=32, and an 8,192-token prefill chunk per rank; cache reuse was disabled.

Across 480 matched indexer-layer calls, the replaced GPU span decreased from 1.524 ms to 1.068 ms at p50, with a 29.9% reduction by paired totals.

Before:
sm100_fp8_paged_mqa_logits (1.522 ms)

After:
_get_k_and_s_triton_kernel + deep_gemm::sm100_fp8_mqa_logits ( <1ms)

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

---

CI States

Latest PR Test (Base): ✅ Run #28550494176
Latest PR Test (Extra): ⏳ Run #28853485756

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[ronhuafeng]

CI triage note for current head eb08e8d5170bf1885400452fc5e25e6fe512cb69:

The actionable blocker I see is only the lint job. black-jupyter reformatted one file:

python/sglang/srt/layers/attention/dsv4/indexer.py

The generated diff is just the max_seqlen_k expression in C4IndexerBackendMixin being collapsed to one line. The adjacent hooks in the same job passed (isort, ruff, codespell, clang-format, registered-test checks, and Rust formatting).

The other red PR Test entries I sampled are gate/aggregate fallout: pr-gate failures and skipped Base/Extra/AMD/NPU/MUSA/XPU/Xeon jobs because maintainer CI labels are not active for this commit. I did not see a GPU/runtime test failure to classify yet. After the black formatting fix, the next useful signal is a normal CI rerun once the PR is labeled/authorized.

[YAMY1234]

/tag-and-rerun-ci

[Fridge003]

/rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp8_h200.py test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

[github-actions]

Results for /rerun-test test/registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py test/registered/models_e2e/test_deepseek_v4_flash_fp8_h200.py test/registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py:

🚀 4-gpu-b200 (2 tests): ✅ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp4_megamoe_b200.py
cd test/ && python3 registered/cp/test_deepseek_v4_flash_fp4_b200_cp.py

🚀 8-gpu-h200 (1 test): ❌ View workflow run

cd test/ && python3 registered/models_e2e/test_deepseek_v4_flash_fp8_h200.py