mirror of
https://github.com/github/awesome-copilot.git
synced 2026-10-13 23:34:25 +00:00
feat(review): assess knowledge uplift in content reviews
Centralize the knowledge-uplift test in the code-review skill and require evidence-backed classifications in the PR Review board. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9082818f-c7db-42c7-add6-c337afb05fdb
This commit is contained in:
co-authored by
Copilot App
parent
9861787546
commit
17ffd2e23b
@@ -25,7 +25,11 @@ export const REVIEW_FIELDS_SCHEMA = {
|
||||
summary: { type: "string", description: "What the pull request actually changes." },
|
||||
repoFit: { type: "string", description: "Repository fit and meaningful value for Copilot users." },
|
||||
compliance: { type: "string", description: "Deterministic checks: frontmatter, naming, README regeneration, manifests." },
|
||||
differentiation: { type: "string", description: "How it differs from existing resources and native model capability." },
|
||||
differentiation: {
|
||||
type: "string",
|
||||
description:
|
||||
"Knowledge-uplift verdict (high / partial / none) with the specific content a capable model would not produce unprompted, plus overlap with existing resources.",
|
||||
},
|
||||
risks: { type: "string", description: "Correctness, security, licensing, or harmful-content concerns." },
|
||||
validation: { type: "string", description: "Evidence the contributor tested or validated the change." },
|
||||
confidence: { type: "string", enum: ["low", "medium", "high"], description: "Confidence in the recommendation." },
|
||||
@@ -60,6 +64,16 @@ function sharedRules(repo, guidancePath) {
|
||||
" - `gh pr diff <number> -R " + repo + "` for the actual change (use `--name-only` first if the diff is large).",
|
||||
" - `gh pr checks <number> -R " + repo + "` when checks are failing, to see which ones and why.",
|
||||
"Judge the submitted result, not assumptions about the tool that produced it. A title ending in 🤖🤖🤖 is a disclosure, not a defect.",
|
||||
"",
|
||||
knowledgeUpliftRules(),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
function knowledgeUpliftRules() {
|
||||
return [
|
||||
"Knowledge uplift — apply the \"Knowledge uplift\" test from the code-review skill to every agent, instruction, skill, prompt, hook, or workflow the PR adds or substantially changes. It is the criterion most often under-applied, so do not skip it for well-formatted PRs:",
|
||||
" - Always start the `differentiation` field with the `high` / `partial` / `none` classification and the quoted lines that justify it, then note any overlap with existing resources.",
|
||||
" - `none` must not be `approve`: use `needs-insight`, or `close` when it is also a near-duplicate. `partial` is usually `request-changes`; name the generic sections to cut in `suggestedComment`.",
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
|
||||
@@ -45,6 +45,10 @@ For every pull request:
|
||||
reason for `request-changes`.
|
||||
- **Duplication** — search the repo for an existing agent/instruction/skill covering the
|
||||
same ground. Near-duplicates are `needs-insight` unless the overlap is total.
|
||||
- **Knowledge uplift** — the most common way a well-formatted PR still fails. Apply the
|
||||
"Knowledge uplift" test from `.github/skills/code-review/SKILL.md` to every content
|
||||
resource. `none` is never `approve` (use `needs-insight`, or `close` if it is also a
|
||||
near-duplicate); `partial` is usually `request-changes`.
|
||||
- **Provenance** — content copied from elsewhere without attribution, or obviously
|
||||
machine-generated filler with no evidence it was tested, is `needs-insight` at best.
|
||||
- **Checks and conflicts** — failing CI or a conflicting branch is `request-changes`,
|
||||
@@ -57,7 +61,9 @@ For every pull request:
|
||||
- `rationale` is one sentence shown on the card. Lead with the deciding factor.
|
||||
- `summary` describes what the PR actually changes, in the reviewer's own words.
|
||||
- `repoFit`, `compliance`, `differentiation`, `risks`, and `validation` map to the
|
||||
SKILL.md criteria — fill in the ones that mattered, leave the rest out.
|
||||
SKILL.md criteria — fill in the ones that mattered, leave the rest out. The exception
|
||||
is `differentiation`: always fill it for content PRs, leading with the knowledge-uplift
|
||||
classification and its evidence.
|
||||
- `checkNext` is for the maintainer: the one thing to verify before acting.
|
||||
- `suggestedComment` is prefilled into the board's comment box, so write it as a comment
|
||||
addressed to the contributor. Be specific, name files and line numbers, and keep the
|
||||
|
||||
@@ -31,7 +31,7 @@ Confirm that a submission addresses a specific GitHub Copilot workflow,
|
||||
technology, domain constraint, or user problem. Flag contributions that:
|
||||
|
||||
- provide generic advice that current models already handle well without
|
||||
meaningful uplift
|
||||
meaningful uplift (see [Knowledge uplift](#knowledge-uplift))
|
||||
- restate an existing resource without a clear differentiator
|
||||
- use broad claims such as doing everything for every project
|
||||
- lack concrete instructions, constraints, examples, or expected outcomes
|
||||
@@ -41,6 +41,49 @@ Paid or commercial services are not automatically unsuitable. Evaluate whether
|
||||
the contribution provides standalone user value and follows the repository's
|
||||
guidance for paid-service submissions.
|
||||
|
||||
## Knowledge uplift
|
||||
|
||||
A resource is only valuable if it changes what the model does. Well-formatted,
|
||||
compliant content can still add nothing. Apply this test to every agent,
|
||||
instruction, skill, prompt, hook, or workflow that a PR adds or substantially
|
||||
changes:
|
||||
|
||||
1. Picture a current frontier model given the same task with no resource
|
||||
loaded. What would it do?
|
||||
2. Identify what this resource makes it do differently. The value is that
|
||||
delta, not the content itself.
|
||||
|
||||
Count as real uplift:
|
||||
|
||||
- facts the model is unlikely to know or get right, such as recent or niche
|
||||
APIs, version-specific breaking changes, and non-obvious gotchas
|
||||
- internal, proprietary, or organization-specific tools and conventions
|
||||
- constraints that deliberately override sensible defaults
|
||||
- bundled scripts, templates, or data that produce deterministic results
|
||||
- multi-step workflows that wire specific tools or MCP servers together
|
||||
- concrete examples of non-obvious expected output
|
||||
|
||||
Do not count as uplift:
|
||||
|
||||
- generic best practices such as clean code, SOLID, DRY, meaningful names,
|
||||
writing tests, or handling errors
|
||||
- restatements of well-known public documentation for mainstream languages and
|
||||
frameworks
|
||||
- persona framing such as "You are an expert X" without new constraints
|
||||
- checklists of behavior models already exhibit by default
|
||||
- long prose that a one-line user request would replace
|
||||
|
||||
Length, polish, and structure are not evidence of value. Quote or cite the
|
||||
specific lines that carry the uplift; if you cannot find any, say so plainly.
|
||||
Classify the result as:
|
||||
|
||||
- **high** — most of the content is non-obvious to a capable model
|
||||
- **partial** — a useful core surrounded by generic filler
|
||||
- **none** — a capable model would behave essentially the same without it
|
||||
|
||||
Do not recommend approval when uplift is **none**. For **partial**, ask the
|
||||
author to cut the named generic sections and expand the specific ones.
|
||||
|
||||
## AI-authored submissions
|
||||
|
||||
A PR title ending in `🤖🤖🤖` is an intentional AI-authorship disclosure from
|
||||
|
||||
Reference in New Issue
Block a user