[Copilot]

Harness retries were visible only in job logs, leaving single-attempt and retried runs indistinguishable in telemetry—including failures with no token accounting.

• Retry accounting: Persist attempt counts, per-attempt exit codes, and structured failure classes across Claude, Codex, and Copilot. Telemetry write failures remain non-fatal.
• Span ownership: Emit gh-aw.harness.attempts and gh-aw.harness.retry_reasons only on the span owning the job’s token usage, including exhausted runs without token data.
• Artifact transport: Include retry records in agent, detection, and compact usage artifacts, with phase isolation and ARC/DinD shared-path collection.

Example attributes for success after two rate-limit failures:

{
  "gh-aw.harness.attempts": 3,
  "gh-aw.harness.retry_reasons": [
    "rate_limit_error",
    "rate_limit_error"
  ]
}

• Fixes Harness retries leave no telemetry: claude_harness.cjs discards the attempt count #71824

[github-actions]

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · gpt56 · ~10 AIC · ⌖ ~6 AIC · ⊞ 14.8K · ◷

Generated by Ponytail Reviewer for #71934

[github-actions]

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

[github-actions]

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · opus55 · ~12 AIC · ⌖ ~37 AIC · ◇ ~3 AIC · ⊞ 1.9K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

[github-actions]

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · haiku45 · ~6 AIC · ⌖ ~9 AIC · ◇ ~3 AIC · ⊞ 9.2K · ◷

🧪 Test quality analysis by Test Quality Sentinel

[github-actions]

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

[github-actions]

Review summary

No actionable blocking issues found in the changed lines. The specialist pass was unavailable because its tool access was rejected; I completed the review from the pre-fetched diff and inspected the retry loop, span ownership, and artifact transport changes directly. No inline comments were warranted.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
_{Comment /review to run again}

[github-actions]

Skills-Based Review 🧠

Applied /tdd and /grill-with-docs (per pr-triage classification: new_feature). Two inline findings posted; otherwise this is a solid, well-tested change.

📋 Key Themes & Highlights

Key Themes

• Architecture consistency: The ARC/DinD consolidation step for harness_retry.json bypasses the existing generateArcDindArtifactConsolidationStep helper instead of extending it (see inline comment on compiler_yaml_post_agent.go).
• Edge-case coverage: send_otlp_span.cjs's harness-retry telemetry guard doesn't explicitly reject records with a null exit_code mid-write, though this may be unreachable in practice given the conclusion-job-only read timing (see inline comment, flagged at moderate confidence).

Positive Highlights

• ✅ harness_retry_runner.cjs uses solid try/finally checkpointing with atomic write-then-rename telemetry persistence.
• ✅ Test coverage across harness_retry_runner.test.cjs, claude_harness.test.cjs, and send_otlp_span.test.cjs is thorough — path resolution, cancellation, soft-deadline, write-failure non-fatal handling, and label filtering are all exercised.
• ✅ Attribute label validation (/^[a-z][a-z0-9_]{0,63}$/) guards against malformed telemetry propagating into OTEL spans.
• ✅ Docs (open-telemetry-attributes.mdx) were updated consistently with the new attributes.

Investigation Note

The diff (882 lines) and pr-triage output were used as primary evidence; cross-engine failure-classification differences (Claude/Codex use multi-class arrays, Copilot uses a single-element array) were investigated and judged to be pre-existing architecture rather than a new inconsistency introduced by this PR, so no inline comment was raised for it.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · ~152 AIC · ⌖ ~16 AIC · ⊞ 10.2K
_{Comment /matt to run again}

[Copilot]

🟡 Changes recommended

Retry records are lost on supported sandbox configurations, and compiler changes leave golden fixtures out of sync.

3 open issues

• Regenerate goldens and Wasm fixtures for new artifact paths · New
• Persist agent telemetry in writable exported storage · New
• Pass isolated workflow data to retry artifact collection · New

What changed in this PR

This PR extends gh-aw’s execution telemetry to preserve harness retry history across Claude, Codex, and Copilot, addressing #71824.

Changes:

• Persist attempt counts, exit codes, and failure classes without making telemetry write failures fatal.
• Attach retry attributes to token-owning spans, including failures without token accounting.
• Carry retry records through artifacts and ARC/DinD collection, with tests and reference documentation.

File	Description
pkg/​workflow/​threat_detection_steps.go	Collects and uploads detection retry records.
pkg/​workflow/​threat_detection_steps_test.go	Checks detection artifact inclusion.
pkg/​workflow/​notify_comment.go	Adds retry records to compact usage uploads.
pkg/​workflow/​notify_comment_test.go	Checks agent and detection usage paths.
pkg/​workflow/​compiler_yaml_post_agent.go	Adds agent artifact paths and ARC/DinD collection.
pkg/​workflow/​compiler_yaml_artifacts.go	Includes retry telemetry in fallback artifacts.
pkg/​workflow/​compiler_artifacts_test.go	Covers artifact paths and ARC/DinD copies.
pkg/​workflow/​awf_arc_dind.go	Exports a shared telemetry directory.
pkg/​workflow/​awf_arc_dind_test.go	Checks the shared-directory export.
pkg/​workflow/​arc_dind_artifacts_test.go	Checks shared-path detection collection.
docs/​src/​content/​docs/​reference/​open-telemetry-attributes.mdx	Documents retry attributes and span ownership.
actions/​setup/​sh/​collect_usage_artifact_files.sh	Copies retry records into compact usage artifacts.
actions/​setup/​sh/​collect_usage_artifact_files_test.sh	Checks collection with and without retry records.
actions/​setup/​js/​send_otlp_span.test.cjs	Covers ownership, phase isolation, and malformed records.
actions/​setup/​js/​send_otlp_span.cjs	Emits retry counts and failure-class attributes.
actions/​setup/​js/​harness_retry_runner.test.cjs	Covers persistence, cancellation, deadlines, and write failures.
actions/​setup/​js/​harness_retry_runner.cjs	Resolves telemetry paths and checkpoints attempts.
actions/​setup/​js/​copilot_harness.cjs	Enables persistence and records Copilot failure classes.
actions/​setup/​js/​codex_harness.cjs	Enables persistence and records Codex failure classes.
actions/​setup/​js/​claude_harness.test.cjs	Covers initial success, retried success, and exhaustion.
actions/​setup/​js/​claude_harness.cjs	Enables persistence and records Claude failure classes.

_{Review effort: Max}

[gh-aw-bot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
2. Review (pkg/workflow/compiler_yaml_post_agent.go:303): [/codebase-design] This ARC/DinD harness-retry copy duplicates the responsibility of generateArcDindArtifactConsolidationStep (pkg/workflow/arc_dind_artifacts.go) with raw inline yaml.WriteString calls instead of extending that helper. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)
3. Review (actions/setup/js/send_otlp_span.cjs:2578): [/tdd] The telemetry read-guard validates attempts and attempt_results.length but never checks that each entry's exit_code is non-null before trusting the record as complete. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)
4. Review (pkg/workflow/compiler_yaml_post_agent.go:71): The new artifact paths change compiled YAML, but the checked-in goldens still omit them, for example pkg/workflow/testdata/TestWasmGolden_AllEngines/claude.golden:1160–1161. Since normalization preserves artifact paths, TestWasmGolden_AllEngines and TestWasmGolden_CompileFixtures will fail. Regenerate and commit the string-API goldens with make update-wasm-golden, and refresh the WasmBinary fixtures with node scripts/test-wasm-golden.mjs --update. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)
5. Review (actions/setup/js/harness_retry_runner.cjs:16): The default agent telemetry path loses retry records on Cloud Hypervisor (not writable) and NVX (not exported), so neither spans nor artifacts receive them. The relevant policies are in pkg/workflow/sandbox.go:339–353 and actions/setup/js/nvx_assets.cjs:9–10. Persist the record under a writable, exported directory such as /tmp/gh-aw/agent/, and update the span readers and artifact collectors together. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)
6. Review (pkg/workflow/threat_detection_steps.go:268): With ARC/DinD and sandbox.agent: false, the early firewall check skips this copy even though the detector still runs in its own AWF sandbox. Its retry record stays in the shared directory, so the detection span and artifacts never receive it. At buildDetectionJobSteps (threat_detection_steps.go:106), pass buildThreatDetectionWorkflowData(data, \"\") into this collection helper instead of the main workflow data. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)
7. Fix failing check build-wasm (FAILURE): https://github.com/github/gh-aw/actions/runs/40223984107/job/121261772547.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 99f1941
Sous-chef work: 07026a55e65d38922bae83849770955ad3f19b3925e0b2ba83030691cae678e5 2b00ecf1f58ca0483155d8799d46d220f26d7c9de72e69f1f5cf5a2c05766542 8acb61a077717e3a460018aacdfde96fd489f77606cd36169e2328d9079b75bb 8ddc5087ca1cfcb26553f34cd57e8ae4d4c129a60e9aafeb1f3d1a1c4f6c9016 be86b6a9c751466a2b7f1881dcf533b525e942439a21df037274aa9a8bb1c22d dd4a83ee943264c5641ece8c9e77f74134a918d7ea8dcdde4135bde91985e2ee
Sous-chef state: c29a3a8af2dffdab5cb84405f5aa81aaf45691f015175d25281d6247f9405986

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · ~6 AIC · ⌖ ~7 AIC · ⊞ 1.1K · ◷
_{Comment /souschef to run again}

[gh-aw-bot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
2. Review (actions/setup/js/send_otlp_span.cjs:2578): Sensitive information disclosure risk: actions/setup/js/send_otlp_span.cjs reads /tmp/gh-aw/{agent,threat-detection}/harness_retry.json and forwards failure_classes into OTLP span attributes (gh-aw.harness.). The values are only regex-filtered and are not bounded/normalized with normalizeHarnessRetryRecord (max attempts / max failure classes) or an enum allowlist, so any process able to write that file during the job can exfiltrate arbitrary data (encoded as many short [a-z0-9_]+ strings) to the external OTEL_EXPORTER_OTLP_ENDPOINT. - Preserve harness retry telemetry in spans and usage artifacts #71934 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef state: 6c91bfc9ff259c9d674c442d8f766022e30adfe08afaf87e9b0be7bf8392ece6

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · ~10 AIC · ⌖ ~18 AIC · ⊞ 1.1K · ◷
_{Comment /souschef to run again}

[Copilot]

@copilot address the following outstanding work in one pass:

1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged latest main via make merge-main in 84dc0e5b4c and fixed the telemetry export review in 1026b9994d. The final local gate passes, including 985 JS tests, impacted Go tests, lint, schemas, and workflow drift checks; secret scanning passed. CodeQL returned no alerts but skipped Go analysis because its database was too large. No CI was triggered. A maintainer still needs to resolve the addressed thread and re-trigger CI for the new head.

[pelikhan]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Merged latest main and resolved the merge conflicts in d295875. I also fixed the session-parser bundle’s missing retry-record dependency; focused tests pass.

[github-actions]

🎉 This pull request is included in a new release.

Release: v1.0.1