feat(review): assess knowledge uplift in content reviews

Centralize the knowledge-uplift test in the code-review skill and require evidence-backed classifications in the PR Review board.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 9082818f-c7db-42c7-add6-c337afb05fdb
This commit is contained in:
Aaron Powell
2026-10-13 15:57:41 +11:00
co-authored by Copilot App
parent 9861787546
commit 17ffd2e23b
3 changed files with 66 additions and 3 deletions
@@ -25,7 +25,11 @@ export const REVIEW_FIELDS_SCHEMA = {
summary: { type: "string", description: "What the pull request actually changes." },
repoFit: { type: "string", description: "Repository fit and meaningful value for Copilot users." },
compliance: { type: "string", description: "Deterministic checks: frontmatter, naming, README regeneration, manifests." },
differentiation: { type: "string", description: "How it differs from existing resources and native model capability." },
differentiation: {
type: "string",
description:
"Knowledge-uplift verdict (high / partial / none) with the specific content a capable model would not produce unprompted, plus overlap with existing resources.",
},
risks: { type: "string", description: "Correctness, security, licensing, or harmful-content concerns." },
validation: { type: "string", description: "Evidence the contributor tested or validated the change." },
confidence: { type: "string", enum: ["low", "medium", "high"], description: "Confidence in the recommendation." },
@@ -60,6 +64,16 @@ function sharedRules(repo, guidancePath) {
" - `gh pr diff <number> -R " + repo + "` for the actual change (use `--name-only` first if the diff is large).",
" - `gh pr checks <number> -R " + repo + "` when checks are failing, to see which ones and why.",
"Judge the submitted result, not assumptions about the tool that produced it. A title ending in 🤖🤖🤖 is a disclosure, not a defect.",
"",
knowledgeUpliftRules(),
].join("\n");
}
function knowledgeUpliftRules() {
return [
"Knowledge uplift — apply the \"Knowledge uplift\" test from the code-review skill to every agent, instruction, skill, prompt, hook, or workflow the PR adds or substantially changes. It is the criterion most often under-applied, so do not skip it for well-formatted PRs:",
" - Always start the `differentiation` field with the `high` / `partial` / `none` classification and the quoted lines that justify it, then note any overlap with existing resources.",
" - `none` must not be `approve`: use `needs-insight`, or `close` when it is also a near-duplicate. `partial` is usually `request-changes`; name the generic sections to cut in `suggestedComment`.",
].join("\n");
}
@@ -45,6 +45,10 @@ For every pull request:
reason for `request-changes`.
- **Duplication** — search the repo for an existing agent/instruction/skill covering the
same ground. Near-duplicates are `needs-insight` unless the overlap is total.
- **Knowledge uplift** — the most common way a well-formatted PR still fails. Apply the
"Knowledge uplift" test from `.github/skills/code-review/SKILL.md` to every content
resource. `none` is never `approve` (use `needs-insight`, or `close` if it is also a
near-duplicate); `partial` is usually `request-changes`.
- **Provenance** — content copied from elsewhere without attribution, or obviously
machine-generated filler with no evidence it was tested, is `needs-insight` at best.
- **Checks and conflicts** — failing CI or a conflicting branch is `request-changes`,
@@ -57,7 +61,9 @@ For every pull request:
- `rationale` is one sentence shown on the card. Lead with the deciding factor.
- `summary` describes what the PR actually changes, in the reviewer's own words.
- `repoFit`, `compliance`, `differentiation`, `risks`, and `validation` map to the
SKILL.md criteria — fill in the ones that mattered, leave the rest out.
SKILL.md criteria — fill in the ones that mattered, leave the rest out. The exception
is `differentiation`: always fill it for content PRs, leading with the knowledge-uplift
classification and its evidence.
- `checkNext` is for the maintainer: the one thing to verify before acting.
- `suggestedComment` is prefilled into the board's comment box, so write it as a comment
addressed to the contributor. Be specific, name files and line numbers, and keep the
+44 -1
View File
@@ -31,7 +31,7 @@ Confirm that a submission addresses a specific GitHub Copilot workflow,
technology, domain constraint, or user problem. Flag contributions that:
- provide generic advice that current models already handle well without
meaningful uplift
meaningful uplift (see [Knowledge uplift](#knowledge-uplift))
- restate an existing resource without a clear differentiator
- use broad claims such as doing everything for every project
- lack concrete instructions, constraints, examples, or expected outcomes
@@ -41,6 +41,49 @@ Paid or commercial services are not automatically unsuitable. Evaluate whether
the contribution provides standalone user value and follows the repository's
guidance for paid-service submissions.
## Knowledge uplift
A resource is only valuable if it changes what the model does. Well-formatted,
compliant content can still add nothing. Apply this test to every agent,
instruction, skill, prompt, hook, or workflow that a PR adds or substantially
changes:
1. Picture a current frontier model given the same task with no resource
loaded. What would it do?
2. Identify what this resource makes it do differently. The value is that
delta, not the content itself.
Count as real uplift:
- facts the model is unlikely to know or get right, such as recent or niche
APIs, version-specific breaking changes, and non-obvious gotchas
- internal, proprietary, or organization-specific tools and conventions
- constraints that deliberately override sensible defaults
- bundled scripts, templates, or data that produce deterministic results
- multi-step workflows that wire specific tools or MCP servers together
- concrete examples of non-obvious expected output
Do not count as uplift:
- generic best practices such as clean code, SOLID, DRY, meaningful names,
writing tests, or handling errors
- restatements of well-known public documentation for mainstream languages and
frameworks
- persona framing such as "You are an expert X" without new constraints
- checklists of behavior models already exhibit by default
- long prose that a one-line user request would replace
Length, polish, and structure are not evidence of value. Quote or cite the
specific lines that carry the uplift; if you cannot find any, say so plainly.
Classify the result as:
- **high** — most of the content is non-obvious to a capable model
- **partial** — a useful core surrounded by generic filler
- **none** — a capable model would behave essentially the same without it
Do not recommend approval when uplift is **none**. For **partial**, ask the
author to cut the named generic sections and expand the specific ones.
## AI-authored submissions
A PR title ending in `🤖🤖🤖` is an intentional AI-authorship disclosure from