Repository navigation
Expand aliases in engine.model-routing.allowed-models instead of emitting literal candidate IDs #70936
Description
Problem
engine.model-routing.allowed-models: [small, sonnet] passes compilation for Copilot, but the compiler emits the aliases as literal AWF model IDs instead of expanding them. AWF then finds no matching live model and fails before the agent starts.
Authors can use aliases for engine.model and supported sub-agent model fields, but cannot currently use them to make a routed shortlist resilient to model retirement.
Reported run and reproduction
The supplied evidence is from private sandbox githubnext/gh-aw-routing-sandbox, run 39656210238, compiled with a development build of gh-aw main 3f6775f, using AWF v0.28.70. The private run was not accessed independently; the compiler and AWF code paths below were verified.
Relevant configuration:
engine:
id: copilot
model-routing:
goal: cost
mode: auto
allowed-models: [small, sonnet]Reported results:
gh aw compileaccepts the workflow without an alias warning.- The lock's AWF configuration contains
"candidateModels": ["github-copilot/small", "github-copilot/sonnet"]. GH_AW_INFO_MODEL_CONFIGrecordsrouting.allowed_models: ["small", "sonnet"].- Startup fails with
Model routing failed (no_route): No available model matches routing.candidateModels, exit code 78, before agent execution.
Expected: the router receives provider-qualified candidate patterns covering the aliases, including github-copilot/*haiku*, github-copilot/gpt-5*mini*, and github-copilot/*sonnet*, rather than literal small or sonnet model IDs.
Diagnosis verified against main
Verified on October 21, 2026 against main at 3f6775f, the same compiler revision reported above.
resolveModelRoutingAllowedModelsinpkg/workflow/model_routing.gotrims provider prefixes, accepts identifier-shaped strings, and prependsgithub-copilot/. It never looks up aliases. Consequentlysmall,sonnet, and a typo such assmalall pass this lexical check.BuildAWFConfigJSONcalls that resolver, intersects candidates with model policy, and emitsapiProxy.routing.candidateModelsfor supported AWF versions/images. The resolver is called fromawf_config_build.go, not defined there.- Existing recursive expansion is available in
expandModelPatterns, backed by the shared alias definitions. The cited goldenaw_info.jsondemonstrates this undersub_agent_models[].patterns. Clarification:resolveModelPolicyForAWFConfigitself merges policy lists; it does not expandsmall. - AWF v0.28.70's
buildRoutingCandidatesfilters the live catalogue throughisModelPermittedByPolicy, then raises this exactno_routeerror if nothing matches. Its policy guard already supports provider-aware glob patterns. This points to missing compiler expansion, not a need for a new AWF alias-matching feature.
Provider and API compatibility
On current main, all four routing engines require GitHub Copilot inference. Therefore the canonical candidate prefix is github-copilot/ for Copilot, Claude, Codex, and Pi, including Claude/Codex with model-provider: github; it is not anthropic/ or openai/ based on the engine name.
Claude and Codex also have existing native API/model-family restrictions. Alias support must respect those restrictions rather than make Codex accept Sonnet. Check compatibility against expanded candidates instead of rejecting a recognized alias solely because its name lacks claude- or gpt-.
Why this matters
On October 19, 2026, the Copilot catalogue dropped five models, invalidating 40 workflow pins, as documented in #70342. That PR and #70652 moved eligible top-level and sub-agent pins to aliases, but routing shortlists still needed exact IDs.
A retirement-resistant explicit shortlist currently requires editing concrete IDs when models disappear. Workflow-wide models.allowed patterns or omitting the shortlist are available, but are not equivalent: model policy also affects sub-agents, whereas routing candidates should constrain only the main router's choice.
Possible solution
Expand known gh-aw aliases in engine.model-routing.allowed-models using the shared alias definitions and existing recursive pattern-expansion logic, rather than introducing a second alias table. Apply this consistently to Copilot, Claude, Codex, and Pi. Retain only the effective routing provider's patterns and canonical prefix; do not leak other providers from multi-provider alias definitions. Preserve exact IDs as exact IDs, with the existing provider qualification and deduplication.
Continue intersecting the expanded candidate set with models.allowed and models.blocked, with blocking taking precedence. Preserve request-policy versus router-shortlist separation. Account for pattern semantics rather than treating an expanded glob as a concrete model name: an exact allow rule within an alias must remain usable, and partial blocking must not widen or erase permitted choices incorrectly. If policy excludes an alias's entire expansion and leaves no candidates, retain the existing policy error and name the authored alias.
Reject literal entries that are neither known aliases nor plausible concrete model IDs during compilation. The error should identify the entry and list valid aliases; smal should not become a runtime no_route. This validation must not require an exact ID to be present in a changing live catalogue.
Record both authored and effective forms. Keep routing.allowed_models in GH_AW_INFO_MODEL_CONFIG / aw_info.model_config, and add the expanded, policy-intersected candidates actually emitted to AWF. Carry them through the existing model.config projection so gh aw audit and the CAO page can show the router's effective pool. Update the metadata builder and sanitization/projection together; adding only a compiler field would otherwise lose it in the collector. Follow the existing snake_case artifact / camelCase event conventions.
Update the model-routing reference, which currently says the shortlist accepts exact names only. Explain aliases, provider-scoped expansion, policy intersection, and remaining API-family restrictions.
Acceptance cases
Each case must have a named test. Use real compiled AWF JSON and the existing aw_info.json / unified-session record shapes, not invented flat routing or audit records.
| Named test | Required assertion |
|---|---|
RoutingAllowedModelsExpandsCopilotAliases |
Copilot [small, sonnet] produces candidate patterns including github-copilot/*haiku*, github-copilot/gpt-5*mini*, and github-copilot/*sonnet*; no literal github-copilot/small or github-copilot/sonnet remains. Assert the complete expected expansion from the shared table. |
RoutingAllowedModelsPreservesMixedExactAndAlias |
[gpt-5.6-luna, small] keeps github-copilot/gpt-5.6-luna exact and expands only small. |
RoutingAllowedModelsScopesAliasesAcrossEngines |
Parameterize the same alias inputs across Copilot, Claude (model-provider: github), Codex (model-provider: github), and Pi at the expansion boundary: every emitted pattern has only the effective github-copilot/ provider. Full-compilation cases must also exercise engine-compatible aliases and preserve Claude/Codex family validation; this is not an authorization for Codex to run Sonnet. |
RoutingAllowedModelsReportsFullyBlockedAlias |
An alias whose entire expansion is excluded by models.blocked, with no remaining candidates, produces the existing policy error and names the authored alias. |
RoutingAllowedModelsIntersectsExpandedPolicy |
Exact and wildcard models.allowed rules narrow an alias correctly; partial models.blocked exclusions preserve permitted choices without allowing blocked models. |
RoutingAllowedModelsRejectsUnknownName |
[smal] fails compilation, identifies smal, and lists valid aliases rather than emitting a literal candidate. |
RoutingModelConfigRecordsAuthoredAndExpandedCandidates |
Decode the real GH_AW_INFO_MODEL_CONFIG, pass through aw_info.model_config and the actual model.config collector projection, and assert authored allowed_models / allowedModels plus expanded effective candidates survive. Candidates must agree with emitted AWF JSON, including policy filtering. |
RoutingAliasesSandboxEndToEnd |
On the PR branch with the PR's compiler build, the reported sandbox workflow routes successfully to an available Haiku or Sonnet model and starts the agent instead of exiting 78. Capture actual selection/outcome records. The sandbox owners will run this on the PR branch; it has not been rerun as part of filing this issue. |
Scope
- Keep the PR to this one fix. Do not change files or behavior unrelated to it.
- Only
engine.model-routing.allowed-modelsis in scope, together with directly required validation, audit metadata propagation, tests, and reference documentation.fallback-modelsaliases and custommodel-env-varengines are separate work. - Before fixing a failing check, compare it with the same check on main. If it also fails on main, do not fix it in this PR; merge main once main is fixed, and note in the PR which checks are known failures on main.
- This applies even if a bot comment, including PR Sous Chef, asks to fix a failing check: fix only failures caused by this PR's changes.
Activity