[Copilot]

Workflow-declared Codex sub-agents were staged only as Markdown. Codex 0.163.0 discovers custom roles only from .toml files, so it never loaded them. The harness resolved and recorded their models, logged a "not applied" warning, and the run still passed (for example daily-arxiv-researcher run 39570856821).

Native role generation (engine_subagent_models.cjs)

• declareCodexSubagentModels is replaced by applyCodexSubagentModels. Before spawn, it writes one $CODEX_HOME/agents/<name>.toml per staged agent.
• Each role file contains:
  • name: the declared block name.
  • description: from frontmatter; required.
  • developer_instructions: the Markdown body without frontmatter; required.
  • model: the existing resolver's concrete bare ID.
  • model_reasoning_effort: written only when the selector carries ?effort=. The value is checked against the efforts Codex supports.
• Agents with no model, or with inherit, get no model key and inherit the main model.
• There is no tools key, because Codex roles inherit the parent's provider, tools and MCP servers.
• Serialization reuses the existing codex_config.cjs TOML serializer.
• The staged Markdown and its workspace copy are not changed.

Example generated role (fixture paper-screener with model: small; instructions abbreviated):

description = "Fast relevance screening of arXiv paper abstracts for GitHub Agentic Workflows"
developer_instructions = "Screen an arXiv paper abstract for relevance to GitHub Agentic Workflows (gh-aw).\n\n…"
model = "gpt-5-mini-2025-08-07"
name = "paper-screener"

Fail closed, matching Claude

The harness exits before CLI spawn when:

• a model is unresolvable. The worker gets a model.resolution record with role, status: "failed", resolved_model: null and the existing cause (model_unavailable or catalog_unavailable).
• a staged file can't be read or parsed, or its description or instructions are blank.
• CODEX_HOME is unset.
• a role file can't be written.

No fallback to the main model. The reserved main rejection and the threat-detection skip are unchanged. The "not applied" warning is removed.

Tests

• The Markdown-only and non-fatal tests are replaced. The new tests use the arXiv run 39394965127 staged files and captured /reflect catalog, with every mutation labelled:
  • codex_declared_small_writes_native_toml_before_spawn
  • codex_declared_dead_model_fails_before_spawn
  • codex_no_declared_agents_is_noop
  • catalog-unavailable
  • quote, backslash and multi-line round-trip
  • missing description
  • main
  • detection phase
• In the harness tests, the stub CLI snapshots $CODEX_HOME/agents when it starts, so assertions reflect the state at spawn time.

Notes for review

• Name source changed: the role name and the resolution role now come from the block (file) name. Previously a frontmatter name: took precedence. This only differs when the two disagree.
• daily-arxiv-researcher still resolves: it uses the openai provider. In main-branch run 39570856821 all four declared models resolved status=exact (gpt-5.6-sol, and gpt-5.4-mini three times), so the fail-closed path should not block it.
• Live smoke not run: smoke_codex_native_subagent_uses_resolved_model still needs an authorized dry-run dispatch on this branch, covering the GitHub/Copilot-through-AWF and OpenAI provider paths. That run is the evidence of native discovery and named invocation; unit tests alone don't show it.
• Docs: the inline sub-agents reference documents the Codex TOML conversion. A patch changeset is included.

• Fixes Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70939

[SivaKesava1]

Review findings (round 2):

Round 2 for this PR (#70945, issue #70939). Review findings to address:

1. Endpoint compatibility (blocking). Codex children inherit the parent's provider and speak /responses only (awf_model_routing.cjs allows Codex only /responses). The resolver doesn't check endpoints, so an alias can resolve to a model Codex can't call: purelock.md declares test-writer with model: large, and main run 39601756922 logged codex sub-agent 'test-writer': model 'large' -> 'claude-sonnet-5.5' (status resolved). With this PR that id is written into test-writer.toml, and the child would fail mid-run because Claude models list only /v1/messages and /chat/completions in the Copilot catalogue. Fix it so a Codex sub-agent only gets a model whose supported_endpoints in /reflect include /responses on the configured provider:
   • aliases resolve among /responses-capable models (so large picks a GPT model for Codex);
   • an exact model without /responses fails closed before spawn, recording model.resolution status: failed with a cause that says why (reuse an existing cause if one fits, otherwise add one and document it in the unified session spec);
   • add named tests for both, using the captured catalogue fixture.
2. List in the PR every Codex workflow on main with inline sub-agents (ab-testing-advisor, api-consumption-report, daily-arxiv-researcher, daily-awf-spec-compiler-surfacing, purelock, spec-enforcer) and what each declared model resolves to with this change, so none of them starts failing at startup unexpectedly.
3. Remove the schemas/logs.schema.json change (drift from Fix health failure clustering and cache evidence reporting #70885 on main; this PR changes no Go code).
4. Make the TOML round-trip test real: parse the generated file with a real TOML parser if the repository already has one as a dependency (don't add a runtime dependency; a dev/test dependency is acceptable only if the repository already uses that pattern), and add cases with triple quotes, control characters, non-ASCII text, CRLF and lines starting with # or [.
   Keep the PR to this one fix. Before fixing a failing check, compare it with the same check on main; if it also fails on main, leave it and note it in the PR. Run make agent-finish, then restore schemas/logs.schema.json to main's version if it regenerates again (that drift comes from Fix health failure clustering and cache evidence reporting #70885 on main, not from this PR). Reply on the PR with what changed in each item.

[Copilot]

Review findings (round 2):

Round 2 for this PR (#70945, issue #70939). Review findings to address:...

All four items are addressed in 7084160.

1. Endpoint compatibility: a Codex sub-agent now only gets a model whose /reflect supported_endpoints include /responses on the configured provider.
   • Aliases resolve only among /responses-capable models. With the purelock catalogue, large now resolves to gpt-6.1-sol instead of claude-sonnet-5.5.
   • An exact model without /responses fails before spawn. An alias that matches only such models fails the same way, without a model refresh. Both record model.resolution with status: failed and a new cause, unsupported_endpoint.
   • The new cause is added to the unified session spec (T-UAS-074, version 2.2.0 with a changelog entry), the session JSON schema, the collector cause list in unified_session.cjs, and the .d.ts types.
   • Named tests use a catalogue fixture captured from run 39601756922 (fixtures/awf-reflect-codex-purelock-39601756922.json): codex_alias_resolves_among_responses_capable_models, codex_exact_model_without_responses_fails_before_spawn and codex_alias_matching_only_models_without_responses_fails_before_spawn.
   • Gap: only the Copilot provider publishes supported_endpoints in /reflect. Providers that publish none, such as the OpenAI direct endpoint, are not filtered.
2. Six-workflow table: it is in the PR description.
   • Will fail at startup: api-consumption-report and daily-awf-spec-compiler-surfacing use the github provider with small. No model in that catalogue matches small and supports /responses, so both will now fail at startup with unsupported_endpoint. They can only start if they declare a /responses-capable model; I left the workflows unchanged to keep the PR to one fix.
   • Still start: ab-testing-advisor, daily-arxiv-researcher, purelock and spec-enforcer.
3. Logs schema: schemas/logs.schema.json is identical to the base again.
4. TOML round-trip test: it now parses the generated file with smol-toml.
   • smol-toml is a test-only devDependency, following the existing ajv and commonmark pattern, and the advisory database check was clean.
   • New cases cover triple quotes, control characters, non-ASCII text, CRLF, and lines starting with # or [.

Checks: the targeted JS tests (547), the typecheck and schema:session:check pass, and CodeQL found nothing.

• make agent-finish stopped at golint on Go files this PR doesn't touch. Running the rest with make -k, the other failures were also in code or setup this PR doesn't touch:
  • Go tests in untouched packages;
  • TestGeneratedOutputSchemasAreCurrent, which fails because the logs schema is kept at base;
  • 5 JS tests that depend on the sandbox;
  • the fix and govulncheck targets.
• I reverted the files the generators rewrote, including the logs schema.
• No PR CI checks were failing.

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · 11.3 AIC · ⌖ 32.1 AIC · ◇ 14.5 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · sonnet50 · 14.2 AIC · ⌖ 15.1 AIC · ◇ 6.65 AIC · ⊞ 12.7K · ◷

🧪 Test quality analysis by Test Quality Sentinel

[github-actions]

✅ Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #70945

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

[github-actions]

One simplification opportunity noted inline.

net: -10 lines possible.

Generated by ✂️ Ponytail Reviewer for #70945 · codex · gpt56 · 21.8 AIC · ⌖ 5.37 AIC · ⊞ 14.8K
_{Comment /ponytail to run again}

[github-actions]

Comment Memory

Peek at saved memory (pr-code-quality-reviewer)

reviewed_at: 2026-10-21T21:55:36Z
review_event: COMMENT
top_themes: incomplete endpoint metadata incorrectly treated as unsupported; stale native Codex role files across reused CODEX_HOME or retries
files_reviewed: actions/setup/js/engine_subagent_models.cjs, actions/setup/js/codex_harness.cjs, actions/setup/js/resolve_model_alias.cjs, actions/setup/js/engine_subagent_models.test.cjs, actions/setup/js/codex_harness.test.cjs
comment_count: 2

Note

This comment is managed by comment memory.

Expand the saved memory block to view or edit the persistent context for this thread.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 6.97 AIC · ⌖ 0.564 AIC · ⊞ 1.6K · ◷
_{Comment /review to run again}

[github-actions]

Verdict

One correctness concern was found in native Codex role staging.

Highlights

The harness writes role files into CODEX_HOME/agents but does not reconcile prior harness-generated files. A reused Codex home or retry can therefore expose stale sub-agents and models that are no longer declared, undermining the fail-closed behavior. An inline comment requests tracking and cleaning only harness-managed role files.

Static review only; tests were not run per review instructions.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · 6.97 AIC · ⌖ 0.564 AIC · ⊞ 1.6K
_{Comment /review to run again}

[github-actions]

Skills-Based Review 🧠

Applied /improve-codebase-architecture and /tdd on the Codex native-TOML sub-agent staging change (fallback triage heuristic used: pr-triage returned a classification, high-impact files were engine_subagent_models.cjs, codex_harness.cjs, engine_subagent_models.test.cjs, resolve_model_alias.cjs, and the new purelock fixture).

📋 Key Themes & Highlights

Key Themes

• One minor nit posted inline: mkdirSync for the roles directory runs once per agent inside the write loop rather than once before it. Harmless given recursive: true, but hoisting it would make the per-agent path read as pure file-writing.

Positive Highlights

• ✅ Fail-closed semantics are implemented thoroughly and symmetrically with Claude: missing CODEX_HOME, unreadable/unparseable staged files, blank description/instructions, unresolvable models, and endpoint-incompatible models all stop the run before CLI spawn, with no fallback to the main model.
• ✅ Real-world fixtures (arXiv run, new purelock run with Copilot-provider endpoint filtering) drive the tests instead of synthetic data, and the harness test snapshots $CODEX_HOME/agents at spawn time so assertions reflect true pre-spawn state — a disciplined regression-test habit per /tdd.
• ✅ TOML round-trip tests explicitly cover quotes, backslashes, CRLF, triple-quote/control-character edge cases, and lines that look like TOML comments/tables, which is exactly the kind of boundary coverage /tdd calls for.
• ✅ The new unsupported_endpoint cause is threaded consistently through the resolver, error messages, unified_session.cjs validation, the .d.ts types, the JSON schema, and the spec changelog — good cross-cutting consistency for a new domain concept.
• ✅ Docs (inline-sub-agents.md) and the unified-session spec are updated in the same PR, keeping the domain language and behavior description in sync with the code.

No blocking issues found; the one comment left is a style suggestion, not a correctness concern.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 105 AIC · ⌖ 15 AIC · ⊞ 10.2K
_{Comment /matt to run again}

[SivaKesava1]

Review findings (round 3):

Round 3 for this PR (#70945, issue #70939). Your round-2 reply shows that merging this PR would make two production workflows fail at startup with unsupported_endpoint: api-consumption-report (metrics-aggregator, history-appender, chart-script-writer) and daily-awf-spec-compiler-surfacing (awf-change-detector, surfacing-gap-evaluator). All five sub-agents declare model: small on the Copilot provider. Since the 19 Oct catalogue change, small → mini has no live /responses-capable member. That breakage is caused by this PR, so fix it here:

1. In those two workflows, change the five sub-agents' model: small to model: gpt-5. That alias (copilot/gpt-5*) resolves among /responses-capable GPT models; gh-aw#70342 used it for the same reason. Recompile only those two lock files with make recompile (or gh aw compile <name>), and check that no other lock file changes.
2. In the PR description's six-workflow table, show every Codex workflow now starting, with the model each sub-agent resolves to against the captured catalogue.
3. Don't change pkg/workflow/data/model_aliases.json. That mini/small has no live GPT member is a separate decision for the maintainers; mention it in the PR as a known follow-up.
4. Don't change anything else.

Keep the PR to this one fix. Before fixing a failing check, compare it with the same check on main; if it also fails on main, leave it and note it in the PR. Regenerated-file drift from main is fine to include. Reply on the PR with what changed.

[Copilot]

🟡 Changes recommended

Model selection and role registration can diverge from workflow declarations, undermining the intended fail-closed behavior.

7 open issues

• Preserve exact model precedence before filtering aliases · New
• Reject non-string model declarations instead of inheriting · New
• Migrate scheduled workflows to endpoint-compatible model selectors · New
• Distinguish unknown endpoint metadata from incompatibility · New
• Propagate and validate reasoning effort from resolved aliases · New
• Detect native-role name collisions before registration · New
• Classify unsupported endpoints separately in audit output · New

1 file not reviewed

• actions/setup/js/package-lock.json: Generated file

What changed in this PR

This PR addresses #70939 by converting workflow-declared Codex sub-agents into native TOML roles and enforcing model resolution before startup.

Changes:

• Generate native roles with instructions, resolved models, and reasoning effort.
• Filter model aliases by endpoint compatibility and fail closed on invalid definitions.
• Add regression coverage, captured fixtures, documentation, and the unsupported_endpoint session cause.

File	Description
docs/​src/​content/​docs/​specs/​unified-agent-session-specification.md	Document the new failure cause and specification version.
docs/​src/​content/​docs/​reference/​inline-sub-agents.md	Explain native Codex roles and endpoint restrictions.
docs/​public/​schemas/​unified-session.schema.json	Accept unsupported_endpoint.
actions/​setup/​js/​unified_session.test.cjs	Cover ingestion of the new cause.
actions/​setup/​js/​unified_session.cjs	Recognize the new resolution cause.
actions/​setup/​js/​types/​unified_session.d.ts	Extend resolution-cause typings.
actions/​setup/​js/​resolve_model_alias.cjs	Add endpoint-incompatibility diagnostics.
actions/​setup/​js/​package.json	Add a TOML parser for tests.
actions/​setup/​js/​package-lock.json	Lock the TOML parser dependency.
actions/​setup/​js/​fixtures/​claude-codex-sub-agents/​purelock-39601756922/​.codex/​agents/​test-writer.md	Capture a representative worker definition.
actions/​setup/​js/​fixtures/​claude_codex_subagent_runs.cjs	Expose purelock fixtures and provenance.
actions/​setup/​js/​fixtures/​awf-reflect-codex-purelock-39601756922.json	Capture provider endpoint capabilities.
actions/​setup/​js/​engine_subagent_models.test.cjs	Test native roles, serialization, and startup failures.
actions/​setup/​js/​engine_subagent_models.cjs	Generate roles and enforce model compatibility.
actions/​setup/​js/​codex_harness.test.cjs	Verify role files exist before spawn.
actions/​setup/​js/​codex_harness.cjs	Gate startup on native-role preparation.
.changeset/​codex-native-toml-subagents.md	Record the patch release behavior.

_{Review effort: Max}

[Copilot]

Review findings (round 3):

Round 3 for this PR (#70945, issue #70939). Your round-2 reply shows that merging this PR would make two production workflows fail at startup with `unsupported_...

Fixed in 018d4e0.

1. Workflows: the five sub-agents now declare model: gpt-5 instead of model: small. That's metrics-aggregator, history-appender and chart-script-writer in api-consumption-report, and awf-change-detector and surfacing-gap-evaluator in daily-awf-spec-compiler-surfacing.
   • Against the captured Copilot catalogue (awf-reflect-codex-purelock-39601756922.json), all five resolve to gpt-5.6-luna, which lists /responses.
   • Running the resolver with small against the same catalogue still gives unsupported_endpoint, so this change is what fixes the startup failure.
   • make recompile changed only those two lock files. In each, the diff is just GH_AW_INFO_SUB_AGENT_MODELS and the body hash.
2. PR description: I rebuilt the six-workflow table; the round-2 table wasn't in the body. It lists every sub-agent's declared model and what it resolves to against the captured catalogues. All six workflows now start:
   • ab-testing-advisor and spec-enforcer: small → gpt-5-mini-2025-08-07
   • api-consumption-report and daily-awf-spec-compiler-surfacing: gpt-5 → gpt-5.6-luna
   • daily-arxiv-researcher: gpt-5.4-mini and gpt-5.6-sol, both exact
   • purelock: large → gpt-6.1-sol
3. model_aliases.json: unchanged. The PR description lists mini/small having no live /responses GPT member as a known follow-up.
4. Nothing else changed: the commit touches only those two workflow sources and their lock files.

Checks:

• No PR CI check was failing, so I had nothing to compare against main.
• make agent-report-progress passed.
• Code review found nothing. The CodeQL scan timed out, so it didn't produce a result.

[gh-aw-bot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
2. Review (actions/setup/js/engine_subagent_models.cjs:849): L679: yagni: Return-only roleFiles API has no production caller. Keep paths local and have tests inspect the roles directory; remove the accumulator and return-field plumbing. - Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70945 (comment)
3. Review (actions/setup/js/engine_subagent_models.cjs:847): This writes the current workflow roles into CODEX_HOME/agents without removing or reconciling previously generated .toml files, so a reused CODEX_HOME can make Codex load stale sub-agents that are no longer declared (or retain an old model/instructions after a retry), violating the fail-closed guarantee and potentially invoking unintended roles. - Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70945 (comment)
4. Review (actions/setup/js/engine_subagent_models.cjs:202): This treats every routing-model entry whose supported_endpoints omits /responses as a definitive incompatibility, but the fixture and catalogue schema explicitly carry candidate_metadata_complete: false for entries whose endpoint data is incomplete. A live Codex-capable model can therefore be filtered out of alias resolution or rejected as unsupported_endpoint solely because the metadata is unknown, causing false startup failures or selecting a different model. - Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70945 (comment)
5. Review (actions/setup/js/engine_subagent_models.cjs:857): [/improve-codebase-architecture] Minor: fs.mkdirSync(rolesDir, { recursive: true }) runs once per agent inside the loop instead of once before it. It's harmless (idempotent, recursive: true), but hoisting it above the loop makes the per-agent write path read as pure file-writing rather than repeated directory setup. - Codex: stage native TOML sub-agents with resolved models and fail closed before startup #70945 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 1c8576a
Sous-chef work: 06eef4558332307e0bf0a5e190e0dc954a820372db2aa5288c80dfb3fd58d70d bcdf19d2a282ddc41a7f3d65cc270025e9930544d48e9d70ee4aa331157245bb bd399ce6e481113df34a15250ba9fa7826740e3b7e5d575114eee76e3824e940 c7be88062f55494f667cb576a006951b49e00b774ad319031a337bbb8f77f10a
Sous-chef state: a1a6bb0926cae1086eef92cd9b20d128c83dfd92db2eb3b764ba5b240355d853

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 4.26 AIC · ⌖ 7.77 AIC · ⊞ 1.1K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged the latest main in 8fc5109c and pushed the review fixes in f8f7691d: removed the return-only role path API, tracked and cleaned only harness-managed role files, preserved untracked user roles, cleaned partial writes, and updated the harness snapshot to read only TOML roles. The endpoint-metadata concern was already covered in 1c8576a3. The impacted JavaScript tests pass; the final gate is blocked only by two Go linter findings present unchanged on main. I did not manually trigger CI.

[SivaKesava1]

Verified at 2b3da8d (fixes gh-aw#70939):

• End to end with this PR's build (private sandbox githubnext/gh-aw-routing-sandbox, run 39700679234, Codex 0.163.0 on Copilot through AWF v0.28.70). The workflow runs Codex with main model gpt-5.6-terra and one inline sub-agent, marker-worker, declared with model: gpt-5.
  • The harness logged model resolution: engine=codex role=marker-worker configured=gpt-5 resolved=gpt-5.6-luna provider=copilot status=resolved and wrote native role /tmp/gh-aw/mcp-config/agents/marker-worker.toml (model: gpt-5.6-luna).
  • Codex discovered the role and delegated. The proxy shows one gpt-5.6-luna /responses 200 request at 00:38:56, between the main agent's five gpt-5.6-terra requests.
  • The parent returned worker returned: CODEX-SUBAGENT-MARKER-70945. Its only commands were the safeoutputs noop call, so the marker came from the child.
• Review rounds:
  • /responses-only resolution, so purelock's large now resolves to gpt-6.1-sol.
  • The small → gpt-5 migration of api-consumption-report and daily-awf-spec-compiler-surfacing; only those two lock files change.
  • An exact id takes precedence over a same-named alias, also after a refresh.
  • A non-string model: fails closed.
  • An endpoint that can't be verified gives catalog_unavailable, not unsupported_endpoint.
  • An effort that comes from an alias is applied and validated.
  • A role-name collision with engine.config or config.toml is refused before spawn.
  • Audit has a dedicated model_endpoint_unsupported finding, with a Go test.
  • The generated-role manifest only reconciles roles the harness created.
• Known limits, not blocking:
  • Providers that publish no endpoint list (OpenAI direct) aren't endpoint-checked.
  • An alias whose effort is rejected stops startup but leaves a resolved record rather than failed.
  • mini/small has no live /responses-capable GPT model since 19 Oct. That's a separate alias-table decision.

[github-actions]

🎉 This pull request is included in a new release.

Release: v0.93.1