Files
awesome-copilot/.github/workflows/setup-labels.yml
T
d82df27473 Phase 3: canvas smoke-test evidence and weekly review metrics (#4184) (#4191)
* Phase 3: canvas smoke-test evidence, safe auto-merge, and weekly review metrics (#4184)

Adds the canvas-smoke-test check with review evidence, a config-gated safe auto-merge workflow (disabled by default), and a weekly review operating metrics workflow, plus maintainer docs.

Refs github/awesome-copilot#4184

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Trigger safe auto-merge sweep after Submission Gate Writer completes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Rename thirdParty identifier to satisfy codespell

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Address Copilot review on canvas smoke test, auto-merge, and metrics

- Bind canvas report writer to the triggering workflow_run PR (base/head repo, ref, SHA)
- Bound PNG inflation by IHDR-derived size and a decoded-bytes budget
- Classify file: URLs as unsafe manifest paths
- Follow CommonJS require() and literal dynamic imports; error on missing
  reachable targets and devDependency-only runtime imports
- Validate removed extension paths and orphaned plugins instead of skipping
- Warn on .py/.rb/.pl scripts
- Auto-merge: support CODEOWNERS teams (fail closed), front matter authors,
  list PRs by head SHA, only fall back to direct merge on clean status,
  drop ineffective check_run trigger
- Metrics: ignore reviews before the PR was ready for review

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Docs: note microsoft/azure-dev-tools as a portable marketplace for the review model

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Auto-merge: only trust submission-gate runs from the Submission Gate Writer

Adds trusted_checks (default submission-gate -> external_id
submission-gate-writer). Any other check run or status with that name on the
head SHA blocks arming, so a PR-defined job cannot satisfy the gate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Harden canvas smoke-test checker

- String-, template-, and regex-aware comment stripping so literals containing // cannot hide imports
- Never follow symlinks or read non-regular files; bounded, contained reads
- Validate plugins whose plugin.json was deleted
- Reject unsafe extension IDs in plugin manifests before materialization

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Phase 3: remove auto-merge, isolate canvas smoke test, harden checker

- Remove safe auto-merge (config, workflow, script, tests, labels) per
  maintainer review; documented as deferred pending a security review.
- Restore validate-canvas-extensions.yml to main and move the smoke test
  to its own canvas-smoke-test.yml workflow scoped to extensions/**.
- Reject duplicate IHDR chunks in preview PNGs.
- Mask string/template/regex literal contents before extracting dynamic
  import() and require() specifiers.
- Resolve package.json imports aliases through the same classifier and
  module graph; fail closed on alias shapes that cannot be analyzed.
- Rename docs to canvas-evidence-and-metrics.md and update references.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Fix canvas smoke and review metrics findings

- Trigger canvas smoke tests for plugin changes while skipping expensive setup when target detection finds no canvas coverage.
- Fail reachable data: imports while preserving warnings for unreachable module data: imports.
- Use base plugin manifests for deleted bundle plugin detection and fail closed when base manifests cannot be read.
- Paginate GraphQL PR review pages before computing weekly review metrics.
- Report workflow run API collection errors as incomplete automation health instead of not found.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Address follow-up canvas review findings

Fail closed on non-literal runtime imports, require canonical PNG termination, restrict comment updates to github-actions[bot], and paginate external-plugin label events.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Powell <me@aaron-powell.com>
2026-10-07 16:36:13 +11:00

185 lines
6.8 KiB
YAML

name: Setup Repository Labels
on:
workflow_dispatch
permissions:
issues: write
jobs:
setup-labels:
runs-on: ubuntu-latest
steps:
- name: Create or update labels
uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
with:
script: |
const labels = {
// Intent labels for PR categorization
'skills': {
color: '1D76DB',
description: 'PR touches skills'
},
'plugin': {
color: '5319E7',
description: 'PR touches plugins'
},
'agent': {
color: '0E8A16',
description: 'PR touches agents'
},
'instructions': {
color: 'FBCA04',
description: 'PR touches instructions'
},
'new-submission': {
color: '006B75',
description: 'PR adds at least one new contribution'
},
'website-update': {
color: '0052CC',
description: 'PR touches website content or code'
},
'external-plugin': {
color: 'FEF2C0',
description: 'Public external plugin submission'
},
'external-plugin-canvas': {
color: '1D76DB',
description: 'External plugin submission includes a canvas extension'
},
'hooks': {
color: 'C2E0C6',
description: 'PR touches hooks'
},
'workflow': {
color: 'BFD4F2',
description: 'PR touches workflow automation'
},
// External plugin intake state labels
'awaiting-review': {
color: 'FBCA04',
description: 'Submission is waiting for automated intake validation'
},
'ready-for-review': {
color: '0E8A16',
description: 'Submission passed intake validation and is ready for maintainer review'
},
'requires-submitter-fixes': {
color: 'D93F0B',
description: 'Submission has quality-gate findings that submitter must fix before maintainer review'
},
'approved': {
color: '1D76DB',
description: 'Submission was approved by a maintainer'
},
'rejected': {
color: 'B60205',
description: 'Submission was rejected by a maintainer'
},
// Re-review labels
'removed': {
color: 'B60205',
description: 'External plugin was removed from the marketplace after re-review'
},
're-review-follow-up': {
color: 'D4C5F9',
description: 'Six-month re-review needs maintainer follow-up before a final decision'
},
'awaiting-approval': {
color: 'FBCA04',
description: 'External plugin awaiting maintainer approval'
},
// Review metrics tracking label (see docs/maintainers/canvas-evidence-and-metrics.md)
'review-metrics': {
color: 'C5DEF5',
description: 'Weekly review operating metrics tracking issue'
},
// Submission gate labels (docs/maintainers/submission-gate.md).
// ready-for-review, requires-submitter-fixes, and approved are shared with intake above.
'awaiting-automation': {
color: 'FBCA04',
description: 'PR is waiting for automated checks to finish'
},
'review-in-progress': {
color: '5319E7',
description: 'PR checks passed and a maintainer review is in progress'
},
'merge-risk:low': {
color: 'C2E0C6',
description: 'Merge risk tier: low (docs, metadata, or small resource update)'
},
'merge-risk:medium': {
color: 'FEF2C0',
description: 'Merge risk tier: medium (new or substantially changed resource)'
},
'merge-risk:high': {
color: 'F9D0C4',
description: 'Merge risk tier: high (automation, scripts, MCP, hooks, or review policy)'
},
// Review routing labels (see .github/review-routing.yml)
'needs-reviewer': {
color: 'D4C5F9',
description: 'Request automated reviewer routing for this PR'
},
'review-overdue': {
color: 'E99695',
description: 'First review is past its due date; a backup reviewer was requested'
},
'review-escalated': {
color: 'B60205',
description: 'Review escalated to the core-maintainer pool'
}
};
let created = 0;
let updated = 0;
let failed = 0;
for (const [name, config] of Object.entries(labels)) {
try {
await github.rest.issues.createLabel({
owner: context.repo.owner,
repo: context.repo.repo,
name,
color: config.color,
description: config.description
});
created++;
core.info(`✓ Created label: ${name}`);
} catch (error) {
if (error.status === 422) {
// Label already exists, try to update it
try {
await github.rest.issues.updateLabel({
owner: context.repo.owner,
repo: context.repo.repo,
name,
color: config.color,
description: config.description
});
updated++;
core.info(`✓ Updated label: ${name}`);
} catch (updateError) {
failed++;
core.error(`✗ Failed to update label ${name}: ${updateError.message}`);
}
} else {
failed++;
core.error(`✗ Failed to create label ${name}: ${error.message}`);
}
}
}
core.info(`
Label setup complete:
- Created: ${created}
- Updated: ${updated}
- Failed: ${failed}
- Total: ${Object.keys(labels).length}
`);
if (failed > 0) {
throw new Error(`Failed to setup ${failed} label(s)`);
}