Skip to content

[HiSparse] fix: reject speculative decoding and mixed chunk at startup - #41347

Merged
ispobock merged 2 commits into
sgl-project:mainfrom
alphabetc1:fix/hisparse-reject-spec-mixed-chunk
Oct 1, 2026
Merged

ispobock merged 2 commits into
sgl-project:mainfrom
alphabetc1:fix/hisparse-reject-spec-mixed-chunk

Conversation

@alphabetc1

@alphabetc1 alphabetc1 commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

validate_hisparse accepts two combinations that crash the scheduler on the first request:

  • Speculative decoding + HiSparse. The draft worker's HiSparseDSATokenToKVPool never gets a HiSparse mapping registered, so the first draft extend dies in translate_loc_to_hisparse_device:
    AttributeError: 'HiSparseDSATokenToKVPool' object has no attribute 'full_to_hisparse_device_index_mapping'.
  • --enable-mixed-chunk + HiSparse. Mixing a prefill into the HiSparse decode batch fails in merge_batch:
    TypeError: can only concatenate list (not "NoneType") to list.

Both were reproduced on a 4-layer DeepSeek-V3.2 config with dummy weights (one H200, greedy, --disable-radix-cache). The control arms are identical to each other, so each crash is specific to the combination:

arm result
no spec, no HiSparse reference
HiSparse only identical to reference
EAGLE/MTP only identical to reference
EAGLE/MTP + HiSparse scheduler crash on the first request
mixed chunk only (concurrent requests) ok
mixed chunk + HiSparse scheduler crash on the first mixed batch

Modifications

Reject both at startup with a clear error, until HiSparse supports them (speculative decoding is in progress in #40670 / #37771, which can drop the check).

Accuracy Tests

  • With this change the speculative + HiSparse launch above exits at startup with ValueError: --enable-hisparse does not support speculative decoding yet.
  • test_server_args.py, test_model_overrides.py: 385 passed.

Checklist

  • Format your code according to the Format code with pre-commit.
  • No new unit tests; this change only adds straightforward startup validation.

CI States

Latest PR Test (Base): ✅ Run #36293131226
Latest PR Test (Extra): ❌ Run #36293131028
Latest PR Test (AMD ROCm 10): ❌ Run #36293131140

@alphabetc1

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Sep 27, 2026
@ispobock
ispobock merged commit c9d6c96 into sgl-project:main Oct 1, 2026
232 of 261 checks passed
@alphabetc1
alphabetc1 deleted the fix/hisparse-reject-spec-mixed-chunk branch October 1, 2026 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants