Repository navigation
Restore safe-output handler and tool-description conformance - #71609
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot resolve the merge conflicts on this branch. |
There was a problem hiding this comment.
🟡 Changes recommended
The new grader dependency is not staged, causing gh aw graders run to fail before evaluation.
4 open issues
What changed in this PR
This PR addresses #71584 by restoring safe-output conformance checks, tool guidance, and diagnostic consistency across gh-aw’s runtime helpers.
Changes:
- Aligns the comment-limit signature check and adds assertions for four conformance rules.
- Labels existing
add_commentconstraints prominently in both tool definitions. - Adds standardized error prefixes, staged-preview notices, and related tests.
| File | Description |
|---|---|
scripts/check-safe-outputs-conformance.sh |
Matches the current comment-limit helper signature. |
scripts/check-safe-outputs-conformance_test.sh |
Adds assertions for four conformance rules. |
pkg/workflow/js/safe_outputs_tools.json |
Emphasizes existing comment constraints. |
actions/setup/js/work_queue_delivery.test.cjs |
Tests the verifier configuration error prefix. |
actions/setup/js/work_queue_delivery.cjs |
Standardizes delivery validation errors. |
actions/setup/js/work_queue_claim_scope.test.cjs |
Tests staged-preview logging and error codes. |
actions/setup/js/work_queue_claim_scope.cjs |
Prefixes scope errors and logs staged guards. |
actions/setup/js/start_mcp_gateway.test.cjs |
Updates gateway diagnostic expectations. |
actions/setup/js/start_mcp_gateway.cjs |
Adds configuration and system error prefixes. |
actions/setup/js/safe_outputs_tools.json |
Mirrors the comment-description update. |
actions/setup/js/safe_outputs_tools_loader.test.cjs |
Tests tool-cloning error codes. |
actions/setup/js/safe_outputs_tools_loader.cjs |
Prefixes cloning and file-write failures. |
actions/setup/js/reconcile_work_queue_run.test.cjs |
Updates the finalizer error expectation. |
actions/setup/js/reconcile_work_queue_run.cjs |
Prefixes queue reconciliation errors. |
actions/setup/js/pr_lifecycle.test.cjs |
Tests staged-preview notices. |
actions/setup/js/pr_lifecycle.cjs |
Logs skipped lifecycle checks in staged mode. |
actions/setup/js/model_pin_check.test.cjs |
Tests missing-configuration error codes. |
actions/setup/js/model_pin_check.cjs |
Prefixes argument and configuration errors. |
actions/setup/js/grader_session.cjs |
Prefixes invalid-session errors. |
actions/setup/js/custom_graders.test.cjs |
Tests invalid-session error codes. |
actions/setup/js/check_version_updates.test.cjs |
Checks the execution-provenance error prefix. |
actions/setup/js/check_version_updates.cjs |
Prefixes compatibility and provenance failures. |
Review effort: Max
| // @ts-check | ||
|
|
||
| const { calculateWorkingSetFromEntries } = require("./working_set_metrics.cjs"); | ||
| const { ERR_VALIDATION } = require("./error_codes.cjs"); |
There was a problem hiding this comment.
Added error_codes.cjs to stageGraderRuntime; the staged grader run test passes. Fix: 441b8a8.
| if (resourceExecution.getStore()?.verification) throw scopeError("Claim resource verification is read-only"); | ||
| if (process.env.GH_AW_SAFE_OUTPUTS_STAGED === "true" || execution?.staged === true) throw scopeError("read-only Claim preview cannot perform resource effects"); | ||
| if (process.env.GH_AW_SAFE_OUTPUTS_STAGED === "true" || execution?.staged === true) { | ||
| logStagedPreviewInfo("Claim resource effects are not performed"); |
There was a problem hiding this comment.
Guarded the preview notice on core.info availability and added a no-logger regression test; the coded Claim scope error remains intact. Fix: 441b8a8.
| if grep -E "\\[(HIGH|MEDIUM|LOW)\\] $finding:" "$OUTPUT" >/dev/null; then | ||
| echo "FAIL: Expected no $finding safe-output conformance findings" | ||
| grep -E "\\[(HIGH|MEDIUM|LOW)\\] $finding:" "$OUTPUT" |
There was a problem hiding this comment.
The conformance test now strips ANSI escape sequences before matching findings, and the regression test passes. Fix: 441b8a8.
…-safe-output-conformance # Conflicts: # scripts/check-safe-outputs-conformance.sh Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Merged |
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: 71ed330
|
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
@copilot resolve the merge conflicts on this branch. |
…-safe-output-conformance # Conflicts: # actions/setup/js/pr_lifecycle.cjs # actions/setup/js/pr_lifecycle.test.cjs # actions/setup/js/safe_outputs_tools.json # actions/setup/js/work_queue_claim_scope.cjs # actions/setup/js/work_queue_claim_scope.test.cjs # pkg/workflow/js/safe_outputs_tools.json Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Merged the latest |
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: 2a99f44
|
|
@copilot resolve the merge conflicts in this pull request |
…-safe-output-conformance # Conflicts: # actions/setup/js/check_version_updates.cjs # actions/setup/js/grader_session.cjs # actions/setup/js/model_pin_check.cjs # actions/setup/js/reconcile_work_queue_run.cjs # actions/setup/js/safe_outputs_tools_loader.cjs # actions/setup/js/start_mcp_gateway.cjs # actions/setup/js/work_queue_claim_scope.cjs # actions/setup/js/work_queue_delivery.cjs Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Updated from latest
Resolved the latest |
| core.error("ERROR: MCP_GATEWAY_DOCKER_COMMAND has incorrect syntax"); | ||
| core.error("Expected: docker run command with image and arguments"); | ||
| core.setFailed(`ERROR: MCP_GATEWAY_DOCKER_COMMAND has incorrect syntax — got: ${dockerCommand}`); | ||
| setFailed(`${ERR_CONFIG}: MCP_GATEWAY_DOCKER_COMMAND has incorrect syntax — got: ${dockerCommand}`); |
There was a problem hiding this comment.
Removed the supplied Docker command from the syntax error and kept the expected format in the diagnostic. Fix: f386a2d.
|
@copilot address the following outstanding work in one pass:
Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI. Sous-chef head: a10832a
|
…-safe-output-conformance Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
@copilot resolve the merge conflicts on this branch. |
…-safe-output-conformance # Conflicts: # pkg/cli/graders_run.go Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Merged latest
Updated from latest |
|
🎉 This pull request is included in a new release. Release: |


Safe-output handlers and tool descriptions had drifted from the conformance specification: required
add_commentconstraints were undisclosed, several handlers lacked standardized error codes, and two staged-mode guards gave no preview notice.IMPORTANT CONSTRAINTSsection to bothadd_commenttool definitions.