Skip to content

Reject placeholder issue titles before consuming create_issue quota - #69994

Merged
pelikhan merged 4 commits into
mainfrom
copilot/safeoutputs-reject-placeholder-titles
Oct 18, 2026
Merged

pelikhan merged 4 commits into
mainfrom
copilot/safeoutputs-reject-placeholder-titles

Conversation

Copilot AI commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

A substantive body allowed placeholder titles such as Test Title to be published and consume the issue allowance, blocking a corrected retry with E002.

  • Title validation: Check the normalized effective title independently of the body. Reject existing trivial probes and exact matches for test title, placeholder title, dummy title, and example title; preserve descriptive titles containing those words.
  • Side-effect ordering: Reject before recording intent, offloading content, or counting quota, with explicit corrective guidance. Preserve body-derived titles, the Agent Output fallback, and PR/comment validation.
  • Regression coverage: Cover normalization, fallback titles, complete and 70,000-character bodies, and max: 1: rejected placeholders leave no append or offload, the corrected issue succeeds, and the next real issue receives E002.
  • Release: Add a patch changeset.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Reject placeholder issue titles before consuming create_issue quota Reject placeholder issue titles before consuming create_issue quota Oct 18, 2026
Copilot AI requested a review from pelikhan October 18, 2026 21:01
@pelikhan
pelikhan marked this pull request as ready for review October 18, 2026 21:31
Copilot AI max review requested due to automatic review settings October 18, 2026 21:31
@pelikhan

Copy link
Copy Markdown
Collaborator

/matt

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Configured title prefixes still allow placeholder submissions to bypass validation and consume quota.

1 open issue
What changed in this PR

Strengthens create_issue validation to reject placeholder titles before recording intent, offloading content, or consuming quota.

Changes:

  • Validate resolved titles independently of body content.
  • Add regression coverage for normalization, fallbacks, large bodies, and quota preservation.
  • Add a patch changeset.
File Description
actions/​setup/​js/​safe_outputs_handlers.test.cjs Adds placeholder rejection and quota regression tests.
actions/​setup/​js/​intent_probe.cjs Rejects known placeholder titles regardless of body content.
.changeset/​reject-placeholder-issue-titles.md Documents the patch release.

Review effort: Max

Comment thread actions/setup/js/intent_probe.cjs Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /tdd — the fix direction (reject placeholder titles before recording intent/offloading/quota) is correct and well tested, but the new whitelist check doesn't fully cover the existing placeholder vocabulary.

📋 Key Themes & Highlights

Key Themes

  • Incomplete generalization: intent_probe.cjs line 105 hardcodes only 4 "<word> title" combinations (test title, placeholder title, dummy title, example title) instead of deriving from the full TRIVIAL_PROBE_VALUES set, so titles like "wip title", "todo title", or "probe title" still bypass validation with a substantive body.
  • Side-effect ordering fix is correctly implemented: the handler now rejects before appendSafeOutput/writeFileSync/quota increment, matching the PR description and verified by the new safe_outputs_handlers.test.cjs test for both normal and oversized bodies.

Positive Highlights

  • ✅ Strong regression coverage for the title-only/body-derived/preserved-descriptive-title cases.
  • ✅ Corrective error message now explicitly tells the agent to replace the placeholder title, which is clearer UX than the prior generic wording.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet50 · 24.2 AIC · ⌖ 16.1 AIC · ⊞ 10.1K
Comment /matt to run again

Comment thread actions/setup/js/intent_probe.cjs Outdated
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (actions/setup/js/intent_probe.cjs:105): Strip the configured title prefix before checking placeholders. With create_issue: { max: 1, title_prefix: "[bot] " }, both [bot] Test Title and [bot]: Test Title pass this guard with a substantive body, although Test Title is rejected. create_issue.cjs uses sanitizeTitle and applyTitlePrefix, so these inputs all resolve to the same published title, [bot] Test Title. The accepted variants can therefore still offload content and consume the allowance, blocking a corrected retry with E002. - Reject placeholder issue titles before consuming create_issue quota #69994 (comment)
  3. Review (actions/setup/js/intent_probe.cjs:105): [/tdd] This hardcoded list of four "<word> title" placeholders doesn't generalize to the rest of TRIVIAL_PROBE_VALUES — titles like "wip title", "todo title", "probe title", "temp title", or "tbd title" still pass through unflagged. - Reject placeholder issue titles before consuming create_issue quota #69994 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 51a7b35
Sous-chef work: 79843f6dba01486e537621252ebc68a85af1e3245f11c815c7c53ea19034ae51 87a249036697145cdf48519396d2bbf6d5a1498a3c8134861622a62bf1c15832
Sous-chef state: 930d6567e62cb6a708b879439d169c118ff96eba4a6d61ebf18a54a6389ea53c

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 6.81 AIC · ⌖ 6.08 AIC · ⊞ 1.1K · ◷
Comment /souschef to run again

Strip configured issue title prefixes before placeholder validation and derive placeholder-title variants from the shared probe vocabulary. Cover prefixed explicit and body-derived titles while preserving offload and quota ordering.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…eject-placeholder-titles

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
@pelikhan
pelikhan merged commit cebf09f into main Oct 18, 2026
2 of 3 checks passed
@pelikhan
pelikhan deleted the copilot/safeoutputs-reject-placeholder-titles branch October 18, 2026 22:35
Copilot AI restored the copilot/safeoutputs-reject-placeholder-titles branch October 18, 2026 22:36
Copilot AI requested a review from gh-aw-bot October 18, 2026 22:36
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v0.92.6

@github-actions
github-actions Bot deleted the copilot/safeoutputs-reject-placeholder-titles branch October 26, 2026 02:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[safeoutputs] Reject placeholder issue titles before consuming create_issue quota

4 participants