* Add submission-gate check, merge-risk tiers, and PR status state machine (#4184)
Phase 2 (enforcement) of github/awesome-copilot#4184:
- submission-gate aggregate required check (reader) with infra vs contribution failure classification
- merge-risk:low/medium/high classification from .github/risk-tiers.yml with tier approval policy
- Submission Gate Writer maintaining state labels and a persistent status comment
- /rerun-checks and /request-review PR commands
- validate-skills and validate-submission-gate workflows, unit tests, and maintainer docs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fix codespell findings and add spelling to submission gate checks
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Harden submission gate per review feedback
- Publish the submission-gate check from the trusted writer via the Checks API
and flag any other submission-gate check run as tampering
- Fail closed on truncated file lists, unscannable diffs, permission lookup
errors, and skipped required checks
- Discard evaluations when the PR head or reviews change mid-run
- Low tier requires a resource owner approval
- Commands must start the comment, matching the workflow filter
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Address #4190 review: command reader/writer, pools, paths, staleness
- Split /rerun-checks and /request-review into a read-only issue_comment
reader and a workflow_run writer that re-reads the comment, PR, and
permission before writing.
- Drop the workflow-security pool; agentic workflows and hooks are
content (medium unless they add scripts or hook commands); the core
pool is core-maintainers.
- Include each check's workflow file and eng scripts in its paths.
- Discard evaluations when the base branch or risk labels change; run
the gate on PR edits (retargeting).
- Correct copies of the writer's check that claim success while the
evaluation fails.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fail closed on lost contributor signal; publish gate check before labels
- A contributor check whose pr-check job succeeded but whose result artifact
is missing or unreadable is now an infrastructure failure, so a lost HIGH
signal can't lower the approval tier.
- Publish the submission-gate check right after stale-state revalidation;
label and comment sync are independent and reported afterwards.
- Revert the self-path added to validate-agentic-workflows-pr.yml's trigger:
that workflow rejects any .github change. Exempt it in the drift test.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fail closed on bad routing config; advisory checks never hold the gate
- loadGateConfig: a missing review-routing.yml still falls back, but a
file that can't be parsed (or whose pools isn't a mapping) now throws
instead of silently relaxing approvals.
- Pending required: false checks no longer keep the PR in
awaiting-automation or the gate check in progress, and the polling loop
doesn't wait on them. They render as pending (advisory).
- PR commands: 👀 marks a command as claimed; 🚀 is written only after
the reply is posted and is the only replay guard, so a run that failed
partway can be retried.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Fix codespell: unparseable -> unparsable
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Switch skill CI checks to vally lint
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Adding Vally to allowed words
* case sensitivity
* Migrate external plugin quality gates from skill-validator to vally lint
Replace the downloaded skill-validator binary with
px @microsoft/vally-cli lint
in the external plugin quality gates pipeline:
- Remove downloadSkillValidator() and SKILL_VALIDATOR_ARCHIVE_URL constant
- Replace uildSkillValidatorArgs() +
unSkillValidatorGate() with
uildVallyLintArgs() +
unVallyLintGate() that run
px vally-cli lint
per resolved skill directory (falling back to the full plugin root when no
specific skill paths can be resolved from plugin.json)
- Rename result keys skill_validator_status / skill_validator_output
to ally_lint_status / ally_lint_output throughout both
ng/external-plugin-quality-gates.mjs and ng/external-plugin-intake.mjs
- Update PR comment markdown to show 'vally lint' instead of 'skill-validator'
- Update CONTRIBUTING.md prose references accordingly
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use @microsoft/vally library directly instead of vally-cli subprocess
Replace the npx-spawned vally-cli process with a direct call to the
@microsoft/vally core library in the external plugin quality gates scripts:
- Add @microsoft/vally as a devDependency in package.json
- Import runLint and LintConsoleReporter from @microsoft/vally
- Replace runVallyLintGate() process spawn with async API call:
- runLint({ rootPath }) returns structured LintResults
- LintConsoleReporter with a Writable capture stream collects
text output without printing to stdout
- Make runExternalPluginQualityGates() async (propagated to
runExternalPluginPrQualityGates() and both main entry points)
- Use Promise.all in runExternalPluginPrQualityGates() for parallel
plugin checks
- Fix remaining skill_validator_status reference in pr-quality-gates
summary string (now vally-lint=...) and YAML workflow table header
- Add 'npm install @microsoft/vally' step to both calling workflows
This removes a layer of indirection (Node -> npx -> CLI -> library)
and replaces it with a direct in-process library call, which is faster,
more reliable, and gives structured access to lint results.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
James MontemagnoandCopilot App
Aaron Powell
Jan Krivanek