mirror of
https://github.com/github/awesome-copilot.git
synced 2026-07-17 03:13:25 +00:00
d47a6c93b7
* Add copilot-pr-autopilot skill Skill that drives any GitHub pull request through repeated rounds of Copilot Code Review until the agent has either resolved every thread or explicitly escalated it to the human. Triggered via GraphQL (no @copilot mention needed), triages every open thread with a fix / decline / escalate rubric, replies and resolves each thread citing the pushed SHA, then re-triggers until HEAD is reviewed with zero threads awaiting the agent's reply. Includes step scripts (01 request-review, 02 check-review-status, 03 list-open-threads, 08 reply-and-resolve, 10 cleanup-outdated), shared library (_lib.ps1) with gh-CLI wrappers (Invoke-Gh, Invoke-GhGraphQL, ConvertFrom-GhJson, Assert-GhReady), reply templates, and reference docs for each step. Repo-agnostic. Requires gh CLI on PATH and repo Triage/Write for full autopilot; external PR authors get single-iteration mode with manual re-trigger via the UI re-request button or a substantive push. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: split per-step references + add recap-gate circuit breaker - Fix #1: give steps 1/7/10 their own reference files (01-request-review.md, 07-commit-push.md, 10-cleanup.md); trim the inline bodies out of orchestration.md so it stays cross-cutting only. - Fix #3: add a recurring round-cap & recap gate to 09-convergence.md — default STOP every 10th round, recap all prior rounds, detect drift (out-of-scope / over-engineering / wrong-direction / belongs-in-separate-PR) with CONTINUE / REVERT-AND-SHIP / HAND-OFF verdicts. Agent reasoning, no new script. - Surface the gate from SKILL.md and orchestration.md; regenerate docs/README.skills.md. Markdown-only change; scripts unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(copilot-pr-autopilot): surface recap gate in decision pseudo-code; clarify Copilot+human convergence and round definition - Inject the round-cap recap gate into the '## Decision: loop back or exit' pseudo-code else-branch so an agent following the code block (not just the prose) runs the STOP-every-10th-round check before looping. - Broaden the 'never terminal' paragraph: non-convergence is driven by a Copilot finding OR a human review comment (this skill handles both); the loop ends only when there are no new comments from either source AND every open thread (Copilot or human) has an agent reply/escalation. - Define a 'round' explicitly as one execution of step 1 (01-request-review), i.e. one Copilot-review trigger — the cap counts review rounds, not tool calls or fix edits. Markdown-only; no script changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * copilot-pr-autopilot: make recap-gate round count deterministic Add 09-review-round.ps1: counts Copilot Code Review submissions straight from the PR's API history (full GraphQL pagination), so the recap-gate trigger is a derived number, not a fallible agent mental tally. This removes the exact failure mode the skill exists to survive — a count drifting across a long run (the real 156-round case). The script reports Round + RecapDue (Round % RecapInterval == 0) only; it never stops the loop or picks the verdict. CONTINUE / REVERT-AND-SHIP / HAND-OFF stays agent reasoning. 09-convergence.md updated to reference the deterministic count while preserving 'no script stops the loop' and 'non-convergence = Copilot finding OR human comment'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Regenerate docs/README.skills.md for copilot-pr-autopilot (add 09-review-round.ps1) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
191 lines
8.0 KiB
Markdown
191 lines
8.0 KiB
Markdown
# Step 4: Triage
|
|
|
|
Sub-agent type: `general-purpose`; budget: **5 min per ≤5 threads**
|
|
(parent batches in waves of ≤5 if step 3 returned more).
|
|
|
|
## Inputs
|
|
|
|
From step 3:
|
|
- Classified thread table — `{ thread_id, file, line, author,
|
|
author_class, severity, summary }` per open thread.
|
|
- Original PR context (description, recent commits to the affected
|
|
files) for re-deriving from principles in oscillation cases.
|
|
|
|
## Return contract
|
|
|
|
Table of rows, one per thread:
|
|
|
|
```
|
|
{ thread_id, action, rationale }
|
|
```
|
|
|
|
Where `action` ∈ `fix` | `decline` | `escalate-to-user` and
|
|
`rationale` is a single line citing the rule from the rubric below
|
|
that fired.
|
|
|
|
## Procedure
|
|
|
|
For each thread, apply the rubric in this file in order:
|
|
|
|
1. **Reviewer-type policy** — `human-or-bot` threads default to
|
|
`escalate-to-user` unless the user explicitly scoped them in.
|
|
2. **ROI vs Risk** — score and decide for `copilot` threads.
|
|
3. **Fix / Decline rule lists** — match the finding pattern.
|
|
4. **Project-specific policy hooks** — research repo conventions
|
|
before applying a generic "fix".
|
|
5. **Conflicting-comments resolution** — if the finding flips a
|
|
prior round's edit, apply the oscillation rules; if the agent is
|
|
about to re-do an edit it already reverted, hard-stop and escalate.
|
|
6. **Escalation rules** — escalate on design-level tradeoffs,
|
|
large/cross-cutting changes, or high-risk/irreversible actions.
|
|
|
|
Return `{ thread_id, action, rationale }` per thread to the parent;
|
|
step 5 consumes only the `fix` rows, step 8 consumes the full table.
|
|
|
|
## Gotchas
|
|
|
|
- **Batch in waves of ≤5 per sub-agent invocation.** Larger batches
|
|
blur the rationale and overrun the 5-min budget.
|
|
- **Human / advanced-security default is `escalate-to-user`.**
|
|
Auto-replying or auto-resolving a human review thread hides
|
|
unaddressed concerns and is socially wrong.
|
|
- **Cite the rule that fired in `rationale`** — one line, concrete.
|
|
"Style nit per rubric §Decline" is fine; "looks small" is not.
|
|
- **The oscillation hard-stop is non-negotiable.** If the agent is
|
|
about to make an edit it already reverted in a prior round, that's
|
|
the unambiguous signature of an oscillation loop — escalate.
|
|
|
|
---
|
|
|
|
# Triage Rubric
|
|
|
|
Decision rubric for whether to fix or decline each Copilot finding.
|
|
The goal is correctness, not appeasement — decline confidently when
|
|
warranted.
|
|
|
|
## ROI vs Risk
|
|
|
|
Score every proposed change on two axes, then decide:
|
|
|
|
- **ROI** = value of the fix MINUS the cost to implement MINUS the
|
|
user's cost to review.
|
|
- **Risk** = blast radius, reversibility, effect on unrelated code.
|
|
|
|
| ROI | Risk | Action |
|
|
|---|---|---|
|
|
| Clear positive | Low | Fix unilaterally. |
|
|
| Marginal | Low | Fix if cheap; otherwise decline with rationale. |
|
|
| Clear positive | High / Irreversible | Propose to the user first. |
|
|
| Marginal | High | Decline. |
|
|
| Negative (over-engineering for hypothetical) | Any | Decline. |
|
|
|
|
## Reviewer-type policy (foundation, not a nit)
|
|
|
|
Each thread's `author` (returned by `03-list-open-threads.ps1`) drives
|
|
who can decide it:
|
|
|
|
| Reviewer | Default action |
|
|
|----------|----------------|
|
|
| `copilot-pull-request-reviewer` / `copilot-pull-request-reviewer[bot]` | Loop-owned — triage with the rubric below. (`03-list-open-threads.ps1` reports the raw `author.login`, which may carry the `[bot]` suffix on some surfaces; match both forms.) |
|
|
| Human reviewer | **Default `escalate-to-user`** unless the user explicitly scoped them into the loop. Auto-replying or auto-resolving a human thread can hide unaddressed concerns and is socially wrong. |
|
|
| `github-advanced-security` / other automated bots | **Default `escalate-to-user`** unless the project has a documented suppression / fix convention you can follow. |
|
|
|
|
## Fix when the finding is...
|
|
|
|
- A **real correctness bug**: use-after-free / lifetime violation,
|
|
race that drops user intent, gating logic that skips legitimate
|
|
transitions, missing link dependency, off-by-one, null deref,
|
|
unhandled error path.
|
|
- A **cross-cutting concern with a clean local fix**: moving a mutex
|
|
one scope up, pulling a duplicated check into a helper.
|
|
- **Documentation / test-plan drift**: PR description claims behavior
|
|
the code no longer matches; comment block describes the wrong
|
|
thing; test-plan checkbox is inverted vs. the code.
|
|
|
|
## Decline when the finding is...
|
|
|
|
- A **purely hypothetical race** requiring cross-class plumbing,
|
|
where actual exposure is negligible. Cite the interleaving you
|
|
ruled out.
|
|
- **Style, naming, or formatting**. Out of scope for the review loop.
|
|
- **Suggestions to add abstractions** ("introduce a strategy pattern",
|
|
"extract an interface") that don't pay for themselves at the
|
|
current scale of the codebase.
|
|
- **Suggestions that contradict an established project convention** —
|
|
consistency with surrounding code is usually more valuable than the
|
|
suggestion in isolation.
|
|
- **Micro-optimizations** in code that is not on a hot path.
|
|
|
|
## Project-specific policy hooks
|
|
|
|
Some findings are decided by **project policy**, not by general
|
|
correctness reasoning. Before applying a generic "fix", **research
|
|
the repo's own conventions first** — `.github/instructions/*.md`
|
|
files (often have an `applyTo` glob that pins them to the changed
|
|
file's path), `.github/skills/`, `AGENTS.md`, `CONTRIBUTING.md`, CI
|
|
config, and recent commits to similar files. Fan out multiple
|
|
`explore` sub-agents when several axes need checking (lint, format,
|
|
spell-check, license header, etc.) — don't invent answers. What
|
|
looks like an obvious fix may violate a project rule:
|
|
|
|
- **Spell-check / dictionary findings.** If the project uses a
|
|
spell-checker (`check-spelling`, `cspell`, `typos`, or similar),
|
|
inspect its config and recent commits to learn the local
|
|
convention (reword the document, add a pattern/regex, extend a
|
|
dictionary/allowlist, use an inline ignore). Follow that
|
|
convention; don't invent a new mechanism.
|
|
- **Lint suppressions / inline ignore directives.** Most projects
|
|
require an inline rationale comment. Bare suppressions get pushed
|
|
back.
|
|
- **License headers / file boilerplate.** Project-specific format,
|
|
often CI-enforced — copy the existing header from a neighbor file.
|
|
- **Test framework, mock library, formatter choices.** Follow the
|
|
convention of the surrounding test files; never introduce a new
|
|
framework because Copilot suggested one.
|
|
|
|
Cite the project's config file or convention in your reply.
|
|
|
|
## Conflicting comments — break oscillation early
|
|
|
|
Failure mode: round N "fixes" what comment A asked for; round N+1
|
|
comment B objects and asks to revert it. Blindly flip-flopping
|
|
ships oscillation and burns rounds.
|
|
|
|
Resolution rules — apply in order, stop at the first that fires:
|
|
|
|
1. **Re-derive from principles.** Pick the side that wins on the
|
|
function's contract + surrounding patterns + user's stated
|
|
preferences — not the latest comment.
|
|
2. **Prefer the position with the explicit rationale.** Concrete
|
|
failure modes (security, correctness, data loss, perf) win over
|
|
stylistic positions.
|
|
3. **Prefer human comments over bot comments** when they directly
|
|
conflict.
|
|
4. **If still ambiguous, escalate to the user.** Summarize both
|
|
positions side-by-side with a recommendation; do not silently
|
|
pick.
|
|
|
|
**Hard stop**: if you are about to make the same edit you
|
|
*reverted* in an earlier round, stop and escalate. That is the
|
|
unambiguous signature of an oscillation loop.
|
|
|
|
## Reply hygiene
|
|
|
|
Every reply — fix or decline — states the reasoning. This makes the
|
|
PR self-documenting and gives the next review visible context.
|
|
Declined findings cite the prior round and explain why the existing
|
|
form is correct, so the next reviewer doesn't raise it again.
|
|
|
|
## Escalation rules
|
|
|
|
Stay in autopilot by default; escalate to the user when:
|
|
|
|
- The finding identifies a **design-level tradeoff** with multiple
|
|
reasonable resolutions and no clear winner.
|
|
- The fix would be **large or cross-cutting** (hundreds of lines,
|
|
new architecture, refactor across modules).
|
|
- The action is **high-risk or irreversible** (force-push, deletion,
|
|
credentials, production).
|
|
|
|
Otherwise, decide and proceed.
|