[hhy-seven]

Summary

The Responses API pre-computed default_max_tokens = context_len - prompt_length.
For multimodal requests prompt_length came from
len(tokenizer.encode(engine_prompt_string)), which only counts the <image>
placeholder — not the engine-side post-expansion token count. So
default_max_tokens was overestimated, and _validate_one_request then rejected
input + default_max_tokens > context_len with a spurious 400 (any model on the
multimodal arch path, e.g. KimiK2.6 sharing KimiK25ForConditionalGeneration).

The expanded count isn't available at the serving_responses layer (image
tokens are only expanded engine-side), which is why a serving-side recompute
can't be made accurate.

Fixes #29287.

Fix

Mirror the Chat Completions route: stop pre-computing default_max_tokens and
pass None, leaving max_new_tokens unset so the scheduler clamps the
budget against the real post-expansion input_len — one source of truth.

• ResponsesRequest.to_sampling_params: default_max_tokens: Optional[int] = None,
  with None-safe handling (max_output_tokens is still honored and capped when
  a concrete default is supplied).
• serving_responses.py: drop the prompt_length / default_max_tokens
  computation and pass None.

This reuses the exact path Chat already uses for unspecified max_tokens
(tokenizer_manager skips the total check when max_new_tokens is None, the
scheduler substitutes a sentinel and clamps to the real input/system limits), so
text Responses requests behave identically to before.

Tests

test_responses_protocol.py::ResponsesSamplingParamsTestCase: None default +
no max_output_tokens leaves max_new_tokens=None; None default +
max_output_tokens=100 → 98; an explicit default_max_tokens=128 still caps
to 126 (backwards compatible). Pass locally.

Per the issue thread @hhy-seven preferred this (option 2); opening as draft for
maintainer confirmation.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

---

CI States

Latest PR Test (Base): 🚫 Run #38012180831
Latest PR Test (Extra): ❌ Run #38012180604
Latest PR Test (AMD ROCm 10): ❌ Run #38012180822

[junetech]

Confirming this fix on a second architecture. I applied the patch to a local checkout and image requests work.

Environment

• Qwen3.8-Flash-Next NVFP4 (Qwen4ExpForConditionalGeneration), TP=2 on 2× RTX PRO 6000 Blackwell (SM120)
• --context-length 262144, EAGLE/NEXTN speculative decoding (num_reserved_tokens = 2)
• Client: Codex, which defaults to the Responses wire API

Before the patch

Request: A 560×560 PNG with a one-line prompt, non-streaming, no max_output_tokens:

Read the text in this image.

Response: 400 invalid_request_error
"Requested token count exceeds the model's maximum context length of 262144
tokens. You requested a total of 262465 tokens: 347 tokens from the input
messages and 262118 tokens for the completion."

One detail that may be worth adding to the issue: the failure does not depend on prompt length.
The input above is 347 tokens.
The overshoot is exactly 321 = 322 image tokens - 1 placeholder token,
so input_expanded + default_max_tokens lands (image tokens - 1) over the context for any image request.
I first hit this on ~184k-token conversations and assumed length was the trigger. It was not.
Three separate failures in my logs all reported the identical total of 263,856 tokens
with inputs of 184,266, 181,762 and 13,302.
Excess is constant 1,712, which was that session's image.

Streaming makes it easy to miss: the 200 and the event-stream header are sent before tokenization, so the request fails inside a 200 stream. My proxy logged seven consecutive failures as successful streamed responses.

After the patch

Same request, unchanged otherwise:

HTTP 200
status : completed
max_output_tokens : null
usage : input_tokens 345, output_tokens 6
output text : "CONTEXT TEST 42" which is the string drawn in the test image

(345 here plus the 2 reserved tokens is the 347 the error reported.)

max_output_tokens: null fix does its job.
The budget is left unset and the scheduler clamps against the post-expansion input length.

Beyond the single repro: 28 /v1/responses requests across two agent clients(codex, pi), multi-turn with images, zero errors.
test/registered/unit/entrypoints/openai/test_responses_protocol.py passes (24 tests, including the 3 added here).

Notes

• Chat Completions was never affected on the same server, which matches the reasoning behind this patch. Mirroring that path makes the budget correct.
• The linked issue [Bug] Responses API overestimates max_output_tokens for multimodal requests, causing 400 error #29287 was auto-closed by the staleness bot rather than by a fix, so it may be worth reopening.

Happy to run further checks on this hardware if that would help move the PR along.

[JustinTong0323]

/tag-and-rerun-ci

[JustinTong0323]

LGTM.

[JustinTong0323]

/rerun-failed-ci