[akhauriyash]

Draft for public review. The K2 Horizon model repositories remain private until release, so please do not merge this PR before the release gate is cleared.

Motivation

Add native SGLang serving support for the IFM K2 Horizon family without checkpoint-provided Python code.

Modifications

• Add native xLLM/K2 Horizon dense, MoE, and MoVA model support.
• Add K2 Horizon reasoning and tool-call parsers with template detection.
• Support non-ambiguous reasoning terminators selected per request from reasoning_effort.
• Preserve reasoning state through Chat Completions, Responses API forwarding, and disaggregated handoff/rebootstrap.
• Add model, parser, grammar, routing, weight-mapping, Responses API, and disaggregation tests.
• Fail during model initialization for unsupported TBO, EPLB/expert remapping, quantized weights, and non-BF16 execution.

The generic serving-core extension is multi-terminator reasoning support. It adds optional, backward-compatible request fields and matching/accounting so one server can handle K2 requests whose reasoning effort selects different end markers. Existing single-terminator parsers keep their existing path.

Cookbook documentation and benchmark cards are in the separate draft PR #37655.

Accuracy Tests

• The public branch is based directly on current sgl-project/sglang main (2d799c28f450a2d83a5bedf2c07699cd32fb3c7a); the runtime diff applied without conflicts.
• Changed-file static checks passed for Python AST, isort, Ruff, merge markers, private keys, and registered-test placement. git diff --check passes after the rebase.
• Focused H200 suite on the validated pre-rebase runtime source: 186 tests and 22 subtests passed; the BF16 MoVA CUDA probe passed with zero maximum absolute error.
• Serving/API validation passed on all six pinned checkpoints, covering base serving, the OpenAI-compatible APIs, reasoning parsing, and tool-call parsing.

Model accuracy results for the K2 Horizon family (scores are percentages):

Model	AIME 2026	HMMT February 2026	LiveCodeBench v6
K2-Horizon-0.9B	48.5	25.8	37.41
K2-Horizon-3.7B	89.69	70.45	59.08
K2-Horizon-7B	90.1	73.3	71.15

Model	GPQA-Diamond	Terminal-Bench 2.1	SciCode
K2-Horizon-32B	82.32	36.6	30.21
K2-Horizon-36B	80.81	58.6	38.89
K2-Horizon-375B	87.3	70.2	42.7

Full GSM8K runtime-validation results are included in the separate cookbook draft.

Speed Tests and Profiling

• Twelve H200 speed points completed: latency and saturation for each of six pinned checkpoints.
• Workload: random token IDs, 512 input and 512 output tokens; 32 prompts at concurrency 1 and 256 prompts at model-sized saturation concurrency.
• Measurements use the base launch commands without parser flags.
• These are validation measurements for the stated H200 recipes, not a comparative performance claim. The cookbook draft contains the final 8,192-input/1,024-output benchmark cards and methodology.

Draft gates

• Run upstream CI on this rebased public commit.
• Keep the model repositories private until the coordinated release.
• Merge the runtime before, or together with, the separate cookbook documentation PR.

Checklist

• Format the changed code according to the SGLang pre-commit guidance.
• Add focused unit, API, disaggregation, and CUDA tests.
• Keep cookbook documentation in a separate draft PR.
• Provide accuracy and speed validation results without making unsupported comparative claims.
• Follow the SGLang code-style guidance.

---

CI States

Latest PR Test (Base): ⏳ Run #33727508015
Latest PR Test (Extra): ⏳ Run #33727507816
Latest PR Test (AMD ROCm 7.2): ⏳ Run #33727508097