Repository navigation
Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70939
Description
Problem
Codex workflows declare inline sub-agents and their models, but gh-aw stages them as Markdown files that the pinned Codex CLI does not load. The harness resolves and records their models without applying either the agents or the models. A workflow can therefore finish green without running its declared workers.
This is the native TOML handoff follow-up explicitly deferred by #70671, which fixed #70667 and merged on October 21, 2026, at 03:07:26 UTC. It is not a request to redo that PR or change alias definitions.
Verified on main
Inspected main at 3f6775f2d6aece54778be6450d5ce710ead62da5.
- The extractor writes Codex inline definitions to
/tmp/gh-aw/.codex/agents/<name>.md; restoration copies them to the workspace's.codex/agents/. - The compiler, rather than the harness, emits
GH_AW_INFO_SUB_AGENT_MODELS. This describes declarations, not proof that Codex loaded or invoked them. - The harness calls
declareCodexSubagentModelsbefore spawning Codex, outside threat detection. That helper resolves the staged models and recordsmodel.resolution, but suppliesfatal: false, warns that they are not applied, and returnsok: trueeven when model resolution fails. The existing reserved-namemainrejection is an exception.
Observed recent run, not just a fixture
daily-arxiv-researcher.md uses Codex and declares four inline agents: preliminary-investigator, paper-screener, and relevance-ranker on openai/gpt-5.4-mini, plus opportunity-extractor on openai/gpt-5.6-sol.
Its October 21, 2026 main-branch run 39570856821, at revision b76697cedc6f0c2f212f7ba7690df9baa7768205, concluded success. In agent job 119124765361, step 36: Execute Codex CLI, the harness logged this warning at 08:41:29.335Z (excerpt; ellipses omit intervening text):
[codex-harness] 2026-10-21T08:41:29.335Z warning: Codex discovers custom agents only from TOML role files ... their declared models are recorded but not applied.
The full line identifies all four files under /tmp/gh-aw/.codex/agents. Immediately following it, each declared model resolves with status=exact, and the harness starts the main process on gpt-5.6-sol. Successful resolution is therefore not evidence of native agent registration.
Precisely: none of these workflow-declared custom roles is available for delegation. This does not mean Codex's built-in/default delegation is universally unavailable; built-in fallback or the main agent doing the work does not satisfy the declared-agent contract.
Native contract at the currently pinned Codex version
DefaultCodexVersion is still 0.163.0. The following findings use OpenAI's rust-v0.163.0 source, not unversioned documentation.
Discovery/location. Codex scans agents/ alongside its enabled configuration layers, including $CODEX_HOME/agents/ and project .codex/agents/. Discovery recursively selects .toml, not .md. A run-local $CODEX_HOME/agents/<name>.toml is a suitable target: gh-aw already manages $CODEX_HOME/config.toml, and its default CODEX_HOME is /tmp/gh-aw/mcp-config. Codex also supports an explicit [agents.<role>] declaration with description and config_file; automatic discovery avoids needing that extra registration.
Sources: pinned role loader, TOML-only discovery, and gh-aw's Codex home selection. Project definitions must belong to an enabled/trusted configuration layer; do not assume that writing a workspace file alone proves discovery.
| Staged information | Native TOML contract |
|---|---|
| Agent name | name = "<declared-block-name>"; required and nonblank for an automatically discovered role, even when the Markdown has no name:. |
| Description | description = "..."; a loaded custom role needs a nonblank description. |
| Instructions | developer_instructions = "...", containing the staged Markdown body, not its YAML frontmatter. Required/nonblank for automatically discovered files. It is not an instructions key. |
| Resolved model | model = "<concrete bare Codex model ID>"; use the existing resolver's normalized result, not small or a provider-qualified selector. |
| Reasoning effort | Optional model_reasoning_effort = "low" (or another supported effort); omitting it inherits the parent's setting. |
| Tools | There is no effective custom-role tools = [...] allowlist. The config parser has a [tools] table for web-search, request-user-input, and update-plan settings, but native role application does not project that table or role-supplied MCP servers into the child. Tools/MCP configuration is inherited; selected capability-disabling [features] overrides are supported. Do not translate Markdown tool names into an invented schema or claim unenforced per-agent restrictions. |
Sources: pinned role-file parser, configuration schema, and bounded role application.
Invocation. The main agent selects the discovered role using the native spawn_agent tool with agent_type equal to its name, supplying the task as message/supported input items. With the default v1 multi-agent surface, this is multi_agent_v1.spawn_agent, for example arguments {"agent_type":"paper-screener","message":"<paper input>"}. Do not use a full-history fork for this example: v1 full-history forks reject an explicit agent-type override. Multi-agent v1 is enabled by default in this pin; v2 is not.
Sources: pinned tool schema, spawn handler, child configuration, and feature defaults.
Model/provider behavior
Yes: a TOML role's model uses the parent's provider and wire API. Codex clones the effective parent configuration, captures its current provider, and then applies the role's model/reasoning/instructions overrides. A role does not choose a new provider merely by choosing a model. The pinned upstream test apply_role_cannot_expand_parent_authority explicitly asserts that model_provider_id, model_provider, model_providers, and mcp_servers remain equal to the parent's, even when the role tries to replace them.
Sources: pinned child configuration, role application, and provider/authority regression test.
For engine.model-provider: github, gh-aw selects GitHub/Copilot inference through AWF. For OpenAI, it selects the OpenAI endpoint. In both AWF configurations Codex itself uses the managed native provider openai-proxy with wire_api = "responses" and the selected provider's proxy base URL; the harness reconciles that URL with /reflect before startup. Child model requests therefore follow the same configured proxy/provider and Codex-side Responses protocol as the main agent. AWF owns any upstream routing/translation; generating a role file is not a per-agent endpoint switch. Resolve and validate the child against the configured provider/catalog, and do not assume an arbitrary cross-provider model will work.
Sources: gh-aw native provider configuration, provider selection, reflect-based endpoint configuration, and pinned Codex wire API definition.
Possible solution
Before Codex starts, consume the same workflow-declared staged definitions used by the existing resolution pass and generate one native TOML role file per agent in the run-local location Codex reads, preferably $CODEX_HOME/agents/. Preserve the declared block name, description, and complete instructions, and write the existing resolver's concrete, provider-normalized model into the file. Use proper TOML serialization so multiline instructions, quotes, and backslashes round-trip. Keep a Markdown copy only if another part of the pipeline needs it; do not edit repository-owned agents or alter other engines' staging.
Make workflow-declared Codex models fail closed, consistent with Copilot and Claude: an unavailable/unresolvable model records model.resolution with the worker's role, status: "failed", and the existing cause (model_unavailable/catalog_unavailable), then stops the harness before CLI spawn. Do not silently inherit the main model, skip the worker, or continue after a native-definition parsing/write failure. Preserve the reserved main protection and the threat-detection exclusion.
Once native definitions are actually applied, remove the current "not applied" warning and replace the tests that deliberately lock in Markdown-only/nonfatal behavior. Keep existing declaration metadata and resolution-record shapes; registration and actual invocation need their own evidence.
Acceptance cases
Use the checked-in real staged arXiv files from run 39394965127, the matching captured /reflect catalog, and the existing harness test layout, rather than hand-written agent-shaped substitutes. Explicitly label fixture mutations such as changing the model selector to small or removing a model from the captured catalog.
Fixtures and current tests: staged definitions, catalog, fixture provenance, and existing Codex tests.
| Named test | Required result |
|---|---|
codex_declared_small_writes_native_toml_before_spawn |
Stage one real agent with its selector explicitly mutated to model: small. Before the mocked CLI starts, exactly one discoverable TOML role exists with the declared name/description, round-tripped instructions, and the concrete model selected by the existing Codex resolver against the captured catalog, not small. Assert the exact fixture-derived ID, not only that a string exists, and a worker model.resolution record with status: "resolved". No "not applied" warning. |
codex_declared_dead_model_fails_before_spawn |
Stage the real paper-screener and remove its gpt-5.4-mini from the complete catalog, including the refresh result, while leaving the main model available. Assert role: "paper-screener", resolved_model: null, status: "failed", cause: "model_unavailable", a nonzero harness outcome, and zero CLI spawns. Catalog-unavailable coverage must also retain the existing failure classification. No fallback to the main model or a built-in worker. |
codex_no_declared_agents_is_noop |
No staged workflow agents means no new TOML files/directories, no native-agent config edits, no worker resolution records, and no change to ordinary Codex startup/provider behavior. |
smoke_codex_native_subagent_uses_resolved_model |
On the implementation PR's published, reviewed branch, a Codex smoke requires its first action to invoke the declared role and receive a fixed worker marker. Verify native role discovery, the actual named invocation and completion, and proxy-served requests attributable to the worker on its resolved model, preferably distinct from the main model. Main-agent assertions, metadata alone, or fallback delegation do not pass. Cover the GitHub/Copilot-through-AWF and OpenAI provider paths, with the inherited provider/Responses invariant. Follow the repository's live-execution gates; do not merge first merely to obtain this evidence. |
This investigation read an existing run; it did not dispatch a new workflow or claim the proposed smoke already passes.
Scope
- Keep the PR to this one fix. Do not change files or behavior unrelated to it.
- Codex sub-agent staging only. Do not change Copilot or Claude sub-agent handling. Do not change aliases, unrelated model pins, provider architecture, or unrelated workflows/checks.
- Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR; merge main once main is fixed, and note in the PR which checks are known failures on main.
- This applies even if a bot comment, including PR Sous Chef, asks to fix a failing check: fix only failures caused by this PR's changes.
Activity