feat(core): consistent oversized-tool-result handling, plus raw-vs-injected size telemetry for every tool #11536

Latest PR verification — 2026-10-07

Formal, open PR #13599 is now at 1e92feee82ed7ad297cac5e65829eeb1761f859c. It preserves the tested production tree and includes the parallel patrol fixes additively. Latest targeted-file receipts cover 1,509 distinct executed passing cases; affected core/CLI builds and typechecks passed. High-pressure native CLI/tmux report and actual JSON MCP discovery/invocation/tmux report verify next-request sizes, complete-output recovery and digests. The reports pin their individual executed commits and controlled-provider boundaries.

MCP deliberately inherits both global character and line limits (defaults 80,000 / 2,000), and explicit values remain independent. This is a proposed default/observability improvement with local verification, not a claim of production billing savings. The PR is unmerged and unreleased; keep this issue open until merge.

Current main verification — 2026-10-07

Verified at 1aba19c87889 using freshly built core/CLI artifacts.

This issue is partially implemented, not complete. PR #11727 has merged and improves producer-side budget ordering. In the freshly built core, a successful read_file ToolCallEvent still exposes content_length but no raw_content_length or injected_content_length fields. Source inspection also confirms that the scheduler recomputes model-facing length after truncation. This is a telemetry-type/configuration check, not an end-to-end metrics-backend run. Current finalizer diagnostics are useful prior work, but they do not prove the requested per-call raw-versus-injected reporting and consistent handling across tools. The issue remains open.

What would you like to be added?

Two coupled changes: make oversized tool-result handling consistent and observable across tools, and report raw-vs-injected result size for every tool call.

A. Oversized-result handling

origin/main already has an extensive pipeline, so this is not a greenfield request. Four ordered layers run in coreToolScheduler before a result reaches history:

1. Persistence gate — maybePersistLargeToolResult() (L6353, called at L5423/L5831). Fires above getTruncateToolOutputThreshold() + GATE_HEADROOM (3000, so 28k by default) and replaces the result with a <persisted-output> stub pointing at tool-results/<callId>.txt. Exempt: GATE_EXEMPT_TOOLS = read_file, read_mcp_resource, enter_plan_mode.
2. Per-tool / global truncation — truncateLlmContent() (L5567) → truncateAndSaveToFile(), budget from maxOutputChars, preview PREVIEW_SIZE_CHARS = 2000 (head/tail), sha256 digest (FULL_OUTPUT_DIGEST_LABEL), idempotency sentinel (TOOL_OUTPUT_TRUNCATED_PREFIX).
3. Combined second pass — L5603, re-bounds at COMBINED_PASS_TOLERANCE_FACTOR (2x) after hook/skill <system-reminder> metadata is appended.
4. Per-message batch budget — applyBatchOutputBudget() (L6403) → finalizeToolResponses() (tools/tool-response-finalizer.ts), reducing the aggregate of parallel calls to DEFAULT_TOOL_OUTPUT_BATCH_BUDGET = 200_000.

Global defaults: DEFAULT_TRUNCATE_TOOL_OUTPUT_THRESHOLD = 25_000, DEFAULT_TRUNCATE_TOOL_OUTPUT_LINES = 1000 (config/config.ts L773-780). Per-tool budgets vary by more than an order of magnitude:

Requests:

1. Re-examine the 500k MCP budget. An MCP result under 500k chars is injected verbatim: no spill, no preview, no event — in practice the largest uncontrolled contributor to injected context, and it is 20x the global default.
2. web_fetch (MAX_CONTENT_CHARS = 100_000, head-only truncateText()) and web_search self-truncate inside execute() with their own wording and no spill-to-file, while shell and MCP self-truncate and spill. The recovery story therefore differs per tool for no stated reason; worth consolidating or at least documenting which mechanism wins.
3. Improve the recovery path when a result does spill. The model sees head+tail only and loses the middle; the ask from the field is to make the stub self-sufficient (total chars/lines, digest, and an explicit "page in with read_file offset/limit" hint) so the model reads the spilled file instead of re-running the tool. truncateKeep is already the per-tool hook.
4. Consider a middle tier between "inject verbatim" and "character-window slice" for structured results (notably MCP JSON): keys/shape + row counts + a sample. A char window can cut JSON mid-token, which is the accuracy complaint behind this request.

B. Telemetry: raw vs injected size, on all four layers

Current state:

• tool_output_truncated covers exactly one of the four layers. logToolOutputTruncated() is emitted only from truncateToolOutput() (tools/truncation.ts), and only inside if (result.outputFile). The persistence gate (persistAndTruncateToolResult) and the batch budget (tool-response-finalizer.ts, tool-result-retention.ts) contain no telemetry calls at all — verified by grep. So the two layers that offload the most content are unobservable, and even layer 2 silently skips the fallback path where the wrapped form is not smaller than the original.
• ToolCallEvent.content_length is post-truncation only. The scheduler captures the raw length first, then deliberately overwrites it (// Recompute AFTER truncation so it reflects the model-facing length, L5659) and again after the batch pass (L6440). The raw number is lost for every call that did not spill.
• contentLength stays undefined for non-string (Part[]) results, so a whole class of MCP results reports no size at all on tool_call.
• No metric carries a size dimension. recordToolCallMetrics attributes are function_name, status, success, decision, tool_type only; TOOL_CALL_COUNT / TOOL_CALL_LATENCY / TOOL_EXECUTION_* have nothing size-related, and endToolExecutionSpan() sets no content-length attribute. Nothing is aggregatable in the metrics backend.
• content_length does not reach the RUM path either — QwenLogger.logToolCallEvent has no size field (it does mirror original_content_length / truncated_content_length for the truncation event).
• ToolResultPersistedEvent is dead code. Declared in telemetry/types.ts L977 with tool_name / bytes_written / output_file / prompt_id, listed in the event union, but never constructed and never logged — there is no logToolResultPersisted and no call site. This is the obvious ready-made hook for layer-1 telemetry.
• Everything is measured in characters; the governance target is tokens.

Requests:

1. On every completed tool call, report raw_content_length and injected_content_length, plus a truncated flag and the applied budget with its source (per-tool vs global fallback vs batch). Compute it for Part[] results too.
2. Emit size telemetry from all four layers — wiring up the existing ToolResultPersistedEvent for the persistence gate is the cheapest first step — so offload rate is measurable per layer, not just for layer 2.
3. Add an estimated token count next to the character count, reusing the estimator the compression pipeline already uses (estimatePartChars, services/compactionInputSlimming.ts) so both agree about the same history.
4. Add a size histogram metric (e.g. qwen_code.tool.result.chars / .tokens) with function_name, tool_type and truncated attributes, so per-tool consumption can be charted and alerted on rather than sampled from logs. Respect the existing metric-cardinality controls.

Already done, do not re-request: analyzeToolResultRetention() diagnostics (toolResultCount, totalChars, largestResultChars, oversizedResultCount) are surfaced by /doctor memory (packages/cli/src/ui/commands/doctorCommand.ts L496/L569). Extending that same view with the layer-1/4 blind spots would be a natural follow-on.

Why is this needed?

Tool results are the dominant controllable contributor to injected context, and the two problems reinforce each other:

• Accuracy. When a result is oversized the model gets a head/tail preview and loses the middle. Reported internally as degrading answer quality; the workaround people fall back to is rewriting the underlying script or skill to emit less, which does not generalise.
• Blindness. We cannot rank tools by injected cost. The raw size is discarded for every call that fits under budget, content_length is post-truncation, and the only size-bearing event fires on one of four layers. Without per-tool raw-vs-injected numbers there is no data to tune the budgets in section A, and no way to tell whether a change helped.

Fixing only one side is not enough: tightening budgets without measurement risks silently degrading accuracy, and adding measurement without a consistent spill path leaves the largest results unbounded.

Additional context

• Read the prior art first. docs/design/2026-08-10-tool-output-offload-preview.md is the authoritative description of the offload/preview state transition and privacy model (required by #4184, mitigation implemented in feat(core): layered tool-output truncation, per-message budget, per-tool limits #4880). Also docs/design/final-tool-response-budget.md — which notes the layers do not share structured state, the root cause of the observability gap in section B — plus docs/plans/final-tool-response-budget-implementation.md and docs/design/tool-use-summary/unified-tool-output.md. FULL_OUTPUT_DIGEST_LABEL came out of #9450 (loop guards fingerprinting persisted output); any new stub format must keep loopDetectionService recognising the sentinels.
• Origin: feedback on tool-result context costs. The request combines spill-to-file and read-on-demand handling with per-tool raw-versus-injected character/token reporting. Extra read round trips and precision loss from partial reads are the concerns addressed by A.3/A.4.
• Related but distinct: PR #10410 preserves the prompt cache for deferred tools. That addresses tool-declaration context cost (deferred loading currently only kicks in above ~10% of model context because loading invalidates the cache), not tool-result size. Not a substitute for this issue.
• All paths, constants and line numbers above were read from origin/main (b1ac3e2970). No existing open issue covers this (searched tool-result truncation / oversized output / result-size telemetry).