mirror of https://github.com/github/awesome-copilot.git synced 2026-10-12 23:04:14 +00:00

Files

T

d82df27473 Phase 3: canvas smoke-test evidence and weekly review metrics (#4184 ) (#4191 )

* Phase 3: canvas smoke-test evidence, safe auto-merge, and weekly review metrics (#4184)

Adds the canvas-smoke-test check with review evidence, a config-gated safe auto-merge workflow (disabled by default), and a weekly review operating metrics workflow, plus maintainer docs.

Refs github/awesome-copilot#4184

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Trigger safe auto-merge sweep after Submission Gate Writer completes

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Rename thirdParty identifier to satisfy codespell

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Address Copilot review on canvas smoke test, auto-merge, and metrics

- Bind canvas report writer to the triggering workflow_run PR (base/head repo, ref, SHA)
- Bound PNG inflation by IHDR-derived size and a decoded-bytes budget
- Classify file: URLs as unsafe manifest paths
- Follow CommonJS require() and literal dynamic imports; error on missing
  reachable targets and devDependency-only runtime imports
- Validate removed extension paths and orphaned plugins instead of skipping
- Warn on .py/.rb/.pl scripts
- Auto-merge: support CODEOWNERS teams (fail closed), front matter authors,
  list PRs by head SHA, only fall back to direct merge on clean status,
  drop ineffective check_run trigger
- Metrics: ignore reviews before the PR was ready for review

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Docs: note microsoft/azure-dev-tools as a portable marketplace for the review model

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Auto-merge: only trust submission-gate runs from the Submission Gate Writer

Adds trusted_checks (default submission-gate -> external_id
submission-gate-writer). Any other check run or status with that name on the
head SHA blocks arming, so a PR-defined job cannot satisfy the gate.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Harden canvas smoke-test checker

- String-, template-, and regex-aware comment stripping so literals containing // cannot hide imports
- Never follow symlinks or read non-regular files; bounded, contained reads
- Validate plugins whose plugin.json was deleted
- Reject unsafe extension IDs in plugin manifests before materialization

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Phase 3: remove auto-merge, isolate canvas smoke test, harden checker

- Remove safe auto-merge (config, workflow, script, tests, labels) per
  maintainer review; documented as deferred pending a security review.
- Restore validate-canvas-extensions.yml to main and move the smoke test
  to its own canvas-smoke-test.yml workflow scoped to extensions/**.
- Reject duplicate IHDR chunks in preview PNGs.
- Mask string/template/regex literal contents before extracting dynamic
  import() and require() specifiers.
- Resolve package.json imports aliases through the same classifier and
  module graph; fail closed on alias shapes that cannot be analyzed.
- Rename docs to canvas-evidence-and-metrics.md and update references.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Fix canvas smoke and review metrics findings

- Trigger canvas smoke tests for plugin changes while skipping expensive setup when target detection finds no canvas coverage.
- Fail reachable data: imports while preserving warnings for unreachable module data: imports.
- Use base plugin manifests for deleted bundle plugin detection and fail closed when base manifests cannot be read.
- Paginate GraphQL PR review pages before computing weekly review metrics.
- Report workflow run API collection errors as incomplete automation health instead of not found.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Address follow-up canvas review findings

Fail closed on non-literal runtime imports, require canonical PNG termination, restrict comment updates to github-actions[bot], and paginate external-plugin label events.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Aaron Powell <me@aaron-powell.com>

2026-10-07 16:36:13 +11:00

3.8 KiB

Raw Blame History

Contributor Reporting (Maintainers) 🚧

This directory contains build scripts and utilities for maintaining the repository.

Build Scripts

`update-readme.mjs`

Generates the main README.md and documentation files from the repository content (agents, prompts, instructions, skills, hooks, collections).

`generate-marketplace.mjs`

Automatically generates .github/plugin/marketplace.json from all plugin directories in the plugins/ folder. This file is used by the GitHub Copilot CLI to discover and install plugins from this repository.

How it works:

Scans all directories in plugins/
Reads each plugin's root plugin.json for metadata
Generates a consolidated marketplace.json with all available plugins
Runs automatically as part of npm run build

To run manually:

npm run plugin:generate-marketplace

`generate-website-data.mjs`

Generates JSON data files for the website from repository content.

`review-routing.mjs`

Reviewer routing and review SLA escalation used by the Review Routing and Review Escalation workflows. Reviewer pools live in .github/review-routing.yml; see docs/maintainers/review-routing.md.

node eng/review-routing.mjs validate      # validate .github/review-routing.yml
node --test eng/review-routing.test.mjs   # unit tests

Review automation

See docs/maintainers/canvas-evidence-and-metrics.md for details.

canvas-smoke-test.mjs — static checks, preview validation, materialization, and install smoke test for canvas extensions (canvas-smoke-test check).
review-metrics.mjs — computes weekly review operating metrics and publishes them to the tracking issue.
lib/review-automation-github.mjs — small GitHub API client used by review-metrics.mjs (uses GITHUB_TOKEN, or the gh CLI locally).

Contributor Tools

contributor-report.mjs — generates a markdown report of merged PRs for missing contributors (includes shared helpers).
add-missing-contributors.mjs — on-demand maintainer script to automatically add missing contributors to .all-contributorsrc (infers contribution types from merged PR files, then runs the all-contributors CLI).

Key notes for maintainers

Reports are generated on-demand and output to reports/contributor-report.md for human review.
The report output is intentionally minimal: a single list of affected PRs and one command to add missing contributor(s).
This repository requires full git history for accurate analysis. In CI, set fetch-depth: 0.
Link: all-contributors CLI documentation

On-demand scripts (not CI)

These are maintainer utilities. They are intentionally on-demand only (but could be wired into CI later).

`add-missing-contributors.mjs`

Purpose: detect missing contributors, infer contribution types from their merged PR files, and run npx all-contributors add ... to update .all-contributorsrc.
Requirements:
- GitHub CLI (gh) available (used to query merged PRs).
- .all-contributorsrc exists.
- Auth token set to avoid the anonymous GitHub rate limits:
  - Set GITHUB_TOKEN (preferred), or GH_TOKEN for the gh CLI.
  - If you use PRIVATE_TOKEN locally, contributor-report.mjs will map it to GITHUB_TOKEN.

Graceful shutdown

contributor-report.mjs calls setupGracefulShutdown('script-name') from eng/utils/graceful-shutdown.mjs early in the file to attach signal/exception handlers.

Testing & maintenance

Helper functions have small, deterministic behavior and include JSDoc comments.
The getMissingContributors function in contributor-report.mjs is the single source of truth for detecting missing contributors from all-contributors check output.

3.8 KiB Raw Blame History