* Add copilot-pr-autopilot skill Skill that drives any GitHub pull request through repeated rounds of Copilot Code Review until the agent has either resolved every thread or explicitly escalated it to the human. Triggered via GraphQL (no @copilot mention needed), triages every open thread with a fix / decline / escalate rubric, replies and resolves each thread citing the pushed SHA, then re-triggers until HEAD is reviewed with zero threads awaiting the agent's reply. Includes step scripts (01 request-review, 02 check-review-status, 03 list-open-threads, 08 reply-and-resolve, 10 cleanup-outdated), shared library (_lib.ps1) with gh-CLI wrappers (Invoke-Gh, Invoke-GhGraphQL, ConvertFrom-GhJson, Assert-GhReady), reply templates, and reference docs for each step. Repo-agnostic. Requires gh CLI on PATH and repo Triage/Write for full autopilot; external PR authors get single-iteration mode with manual re-trigger via the UI re-request button or a substantive push. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: split per-step references + add recap-gate circuit breaker - Fix #1: give steps 1/7/10 their own reference files (01-request-review.md, 07-commit-push.md, 10-cleanup.md); trim the inline bodies out of orchestration.md so it stays cross-cutting only. - Fix #3: add a recurring round-cap & recap gate to 09-convergence.md — default STOP every 10th round, recap all prior rounds, detect drift (out-of-scope / over-engineering / wrong-direction / belongs-in-separate-PR) with CONTINUE / REVERT-AND-SHIP / HAND-OFF verdicts. Agent reasoning, no new script. - Surface the gate from SKILL.md and orchestration.md; regenerate docs/README.skills.md. Markdown-only change; scripts unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(copilot-pr-autopilot): surface recap gate in decision pseudo-code; clarify Copilot+human convergence and round definition - Inject the round-cap recap gate into the '## Decision: loop back or exit' pseudo-code else-branch so an agent following the code block (not just the prose) runs the STOP-every-10th-round check before looping. - Broaden the 'never terminal' paragraph: non-convergence is driven by a Copilot finding OR a human review comment (this skill handles both); the loop ends only when there are no new comments from either source AND every open thread (Copilot or human) has an agent reply/escalation. - Define a 'round' explicitly as one execution of step 1 (01-request-review), i.e. one Copilot-review trigger — the cap counts review rounds, not tool calls or fix edits. Markdown-only; no script changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * copilot-pr-autopilot: make recap-gate round count deterministic Add 09-review-round.ps1: counts Copilot Code Review submissions straight from the PR's API history (full GraphQL pagination), so the recap-gate trigger is a derived number, not a fallible agent mental tally. This removes the exact failure mode the skill exists to survive — a count drifting across a long run (the real 156-round case). The script reports Round + RecapDue (Round % RecapInterval == 0) only; it never stops the loop or picks the verdict. CONTINUE / REVERT-AND-SHIP / HAND-OFF stays agent reasoning. 09-convergence.md updated to reference the deterministic count while preserving 'no script stops the loop' and 'non-convergence = Copilot finding OR human comment'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Regenerate docs/README.skills.md for copilot-pr-autopilot (add 09-review-round.ps1) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
14 KiB
name, description
| name | description |
|---|---|
| copilot-pr-autopilot | Copilot left 14 review comments on your PR — half are nits. Hours of fix → reply → resolve → re-request, and each round lands MORE comments. This skill runs loop engineering: auto-triggers Copilot Code Review via GraphQL (no @copilot mention), triages every open thread (Copilot, humans, advanced-security) with a fix / decline / escalate rubric, dispatches parallel fix sub-agents that obey the repo build/test/lint conventions, commits per iteration, replies+resolves citing the pushed SHA, then re-triggers until HEAD is reviewed with zero threads awaiting the agent's reply (remaining open threads are explicit hand-offs to the human — escalated declines, design tradeoffs). You merge a clean PR; the bot runs it. Trigger phrases: "address copilot comments", "run a copilot review loop", "fix this PR", "iterate on copilot feedback". Repo-agnostic, gh CLI + PowerShell. Full autopilot needs repo Triage/Write; external PR authors get single-iteration mode plus manual re-trigger (UI 🔄 or substantive-commit push). |
Copilot PR Autopilot
Drive any GitHub pull request through repeated rounds of Copilot code
review until the agent has done its job — every Copilot finding has
a reply from the agent (fix-acknowledgement, decline-with-rationale,
or explicit escalate-to-user hand-off). Remaining open threads, if
any, are deliberate hand-offs to the human merge owner — they're
not loop failures. Repository-agnostic — works on any repo that has
Copilot Code Review enabled, run from a machine with gh CLI
installed and authenticated (see Prerequisites).
When to Use This Skill
- The user asks to "request Copilot review" or "run a Copilot review loop" on a PR.
- A PR is functionally complete and the user wants a final correctness pass via repeated automated review rounds.
- A previous Copilot review on the PR has left open threads that need triage, fixing, replying, and resolving.
When NOT to Use This Skill
- The PR is still under active design — wait until the structure is stable; otherwise findings churn round-over-round.
- The user wants human reviewer feedback, not Copilot's.
Prerequisites
ghCLI installed and authenticated against the target repository.- PowerShell on PATH — Windows PowerShell 5.1+ (
powershell.exe) or PowerShell 7+ (pwsh). Both are tested. - Copilot Code Review is the primary use case (
01-request-review.ps1uses GraphQLrequestReviewsByLoginto trigger Copilot). It is NOT a hard requirement — if01-request-review.ps1fails because Copilot isn't enabled on the repo / account, the agent can still drive existing review threads (human, advanced-security, etc.) to completion by running steps 3–8 once as a single iteration; just skip the trigger + wait. There is no auto-detect for "Copilot unavailable" — the agent makes that decision after the trigger fails (the script can't reliably tell "Copilot disabled" from "Copilot enabled but not yet triggered" from API state alone).
Permissions: who can run the full loop
The full multi-round autopilot (steps 1 → 9 → 1) needs Triage or Write permission on the target repo, because GitHub's only public API for adding the Copilot bot as a reviewer (requestReviewsByLogin) is gated on that permission. Verified against the public REST + GraphQL surface in this PR's commit history — there is no public-API path for bot reviewers without write permission.
| You are… | What works |
|---|---|
| Repo collaborator with Triage / Write | Full loop: 01 triggers Copilot, 02 waits, 04–08 triage / fix / reply, loop back to 01. Hands-off. |
| External PR author (no write permission) | 01 will throw a clear actionable error. Use -SingleIteration mode: address all current findings in one pass, then either click the UI 🔄 next to Copilot, or push a substantive commit (the synchronize event auto-triggers Copilot on most repos). Then re-run 02 to verify. |
In single-iteration mode the loop's convergence boolean is Converged: true iff OpenThreadsAwaitingReply == 0 (the agent's side is done). The maintainer-side re-trigger then drives any additional rounds.
Every script dot-sources scripts/_lib.ps1 which
runs Assert-GhReady on load: if gh is missing OR gh auth status
fails, the script halts before any work with a single actionable
error message naming the install command and gh auth login. The
agent should surface that message to the user verbatim and stop the
loop — do not retry or work around it.
Step-by-Step Workflow
The loop: steps 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9, then back to step 1 if
Converged: false. Repeat the 1→9 round until step 9 returnsConverged: true; only then run step 10 once and calltask_complete. At every 10th round, the parent runs the round-cap recap gate before looping back — recap all prior rounds and stop if the loop has drifted out of the PR's original scope.
Each round runs steps 1–9; step 10 is a one-time cleanup after convergence. The parent agent coordinates; every sub-agent step runs in a fresh context with a bounded budget. Cross-cutting protocol (time-boxing, extension, single-iteration fallback): orchestration.md.
- Request review (parent) — see 01-request-review.md
- Wait for review (sub-agent, 20-min cap) — see 02-wait.md
- List + categorize open threads (sub-agent, 5 min) — see 03-list-threads.md
- Triage (sub-agent, 5 min per ≤5 threads) — see 04-triage.md
- Fix (sub-agents, parallel max 5, 5 min each) — see 05-fix.md
- Build + test per repo conventions (sub-agent, 10 min) — see 06-build-test.md
- Commit + push (parent) — see 07-commit-push.md
- Reply (always) + resolve (conditional) (sub-agent drafts, parent posts) — see 08-reply-resolve.md
- Convergence verify (sub-agent, 3 min) — see 09-convergence.md
Converged: false→ loop back to step 1 for another round (re-trigger, wait, list, triage, fix, push, reply, re-check). Each round addresses Copilot's findings on the previous round's HEAD; the loop terminates as soon as Copilot has nothing new to say AND every open thread has a reply from the agent.Converged: true→ exit the loop, run step 10 once, calltask_completewith the proof.- Every 10th round (10, 20, 30…) → run the round-cap recap gate before looping back. Recap ALL prior rounds against the PR's original scope and pick a verdict: CONTINUE, REVERT-AND-SHIP (drop drifted commits, ship the in-scope ones), or HAND-OFF (escalate to the user). This is the circuit breaker that stops a runaway bot-review loop.
- Cleanup outdated (parent, post-convergence, once) — see 10-cleanup.md
Convergence is computed by scripts/02-check-review-status.ps1 as a single Converged: true boolean. Do not call task_complete until it returns true; print the proof (HeadOid, LatestCopilotReview.commitOid, submittedAt) in the completion message.
Gotchas
The bundled scripts enforce the hard correctness invariants (trigger landing via copilot_work_started event id, Converged requiring HEAD-match + zero-awaiting + at-HEAD review, single-iteration fallback semantics, PR-state guard). Trust them — don't re-derive. The notes below cover decisions the scripts can't make for you:
- Reply to every open thread; resolve only when the loop owns the disposition. For
fixanddeclinethreads, reply + resolve. Forescalate-to-userthreads, reply with the analysis but leave the thread OPEN (08-reply-and-resolve.ps1 -NoResolve) so the human merge owner can act on it. See 08-reply-resolve.md. - Copilot threads are loop-owned; human / advanced-security / other-bot threads default to
escalate-to-user. Auto-resolving a human review thread can hide unaddressed concerns. See 04-triage.md for the rubric. - One focused commit per round, not one per PR. Bundling rounds destroys the audit trail of which finding drove which change and breaks
git bisect. See 07-commit-push.md. - Build/test/lint with the repo's own commands (per its
CONTRIBUTING/AGENTS/README/package.json/Makefile) before pushing a fix. Discovery procedure: 06-build-test.md. - Push back with written rationale when a Copilot finding would over-engineer the design for a hypothetical edge case. Auto-accepting every suggestion erodes the design — see the
declinepath in 04-triage.md. - Scripting traps (
gh api graphql -Ftype-coercion,git stash push -mpositional parsing, the three GraphQL traps for the reviewer mutation) are documented in references/api-quirks.md. Read before modifying any script.
Troubleshooting
| Issue | Solution |
|---|---|
Script throws prerequisite missing — gh CLI is not on PATH |
Install gh (winget install GitHub.cli on Windows; brew install gh on macOS; package manager on Linux; or download from https://cli.github.com). Then gh auth login. Surface the message to the user and STOP the loop — do not retry. |
Script throws prerequisite missing — gh CLI is not authenticated |
Run gh auth login. STOP the loop until the user completes auth. |
Trigger fails or no copilot_work_started event lands |
Push a substantive (non-whitespace) commit — auto-assign on synchronize is the most reliable trigger. Persistent failure indicates Copilot Code Review may not be enabled on the repo / account (check repo Settings → Code & automation → Copilot, or account-level Copilot Pro/Pro+). |
| No new review after waiting ~10 min | Quiet-period after recent dismissal or trivial-diff suppression. Push a substantive commit and retry. Do not blindly re-run 01-request-review.ps1 — it reports InFlight while Copilot is still a requested reviewer. |
| Outdated-but-unresolved threads in the open list | Expected: unresolved state is the source of truth. Reply + resolve them like any other open thread. 10-cleanup-outdated.ps1 is only a final safety net. |
| Unsure whether to fix or decline a finding | See references/04-triage.md. |
| Need a reply phrasing for "fixed", "declined", or "drift" | See the templates under templates/ — reply-fix.md, reply-decline.md, reply-drift.md, reply-partial.md. |
References
- references/orchestration.md — cross-cutting loop control: time-boxing & extension protocol, sub-agent delegation map, single-iteration fallback, and loop-wide notes.
- Per-step contracts (one
NN-*.mdper step): references/01-request-review.md (parent), references/02-wait.md, references/03-list-threads.md, references/04-triage.md (includes the fix-vs-decline rubric), references/05-fix.md, references/06-build-test.md, references/07-commit-push.md (parent), references/08-reply-resolve.md, references/09-convergence.md (includes the round-cap recap gate), references/10-cleanup.md (parent). - references/api-quirks.md — verified GitHub API behavior, dead-ends, and the GraphQL traps for the reviewer mutation.
- Templates (one per reply type): templates/reply-fix.md — accepted-fix pattern; templates/reply-decline.md — declined-with-rationale pattern; templates/reply-drift.md — PR-description / comment / test-plan drift acknowledgement; templates/reply-partial.md — partial fix with deferred follow-up. Cross-cutting reply guidance and anti-patterns live in references/08-reply-resolve.md.
- scripts/_lib.ps1 — shared helpers (
Invoke-Gh,Invoke-GhGraphQL,Resolve-RepoCoords); dot-sourced by every script. - scripts/01-request-review.ps1 —
trigger Copilot review and verify pickup via the
copilot_work_startedevent. - scripts/02-check-review-status.ps1 —
single-shot snapshot of the PR's Copilot review state; emits
Converged: trueonly when all three conditions hold. - scripts/03-list-open-threads.ps1 — every unresolved PR review thread from all reviewers (Copilot, humans, github-advanced-security, etc.).
- scripts/08-reply-and-resolve.ps1 — post a reply and resolve in one call.
- scripts/10-cleanup-outdated.ps1 — safety net for outdated Copilot threads.