Migrate plugins and canvas extensions to Agent Plugins spec (#2546)

* feat: migrate plugins and extensions to Agent Plugins v1.0.0 spec - Add \ to all 69 curated plugin manifests - Migrate all 18 extension manifests: add \, move logo into xtensions.com.github.copilot.logo namespace, remove top-level logo and string xtensions: '.' - Update eng/validate-plugins.mjs: require \, validate namespace-keyed extensions object for canvas extensions, widen name pattern to allow dots (spec §5.5, max 64 chars) - Update eng/materialize-plugins.mjs: emit spec-clean served manifests (only spec fields: \, name, version, description, author, homepage, repository, license, keywords, extensions) - Update eng/generate-website-data.mjs: read logo from namespace with fallback to top-level logo for compatibility - Update eng/create-plugin.mjs: scaffold emits \ - Add .github/workflows/validate-plugins.yml: blocking CI for PRs touching plugins/** or extensions/** - Add spec compliance check to external plugin quality gates: non-blocking warnings with ✅/⚠️/🛑 emoji legend - Update AGENTS.md: document new extension manifest shape, add \ to plugin checklist Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * refactor: consolidate canvas extension plugins - Move all extension plugin manifests from extensions/<name> to plugins/<name> - Keep extensions/<name> as reusable source only - Remove standalone extension discovery from marketplace and website plugin catalogs - Auto-bundle same-name extension sources during materialization - Add build-only extensions.json references for sharing extensions across plugins - Remove x-awesome-copilot extension metadata support - Update validation and contributor documentation Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * feat: add canvas extension scaffolding skill - Add repo-local skill for creating canvas extension sources - Generate spec-compliant plugin manifests under plugins/ - Support registering reusable extensions with multiple plugins - Remove guidance for extension-local plugin manifests and custom fields Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * fix: align extension namespaces with current guidance - Use each extension ID as its manifest namespace key - Update validation and website generation to resolve extension-specific namespaces - Upsert plugin validation PR comments using the existing repository pattern Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * fix: use Copilot extension namespace - Adopt com.github.copilot for all canvas extension manifests - Require the namespace during validation and website generation - Update extension scaffolding guidance Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * docs: regenerate plugin catalog after merge Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * refactor(plugins): move manifests to plugin roots Use root plugin.json manifests and namespaced extension directories throughout local tooling, validation, generation, and contributor documentation. Restore materialize-plugins.mjs line breaks so the source remains readable in GitHub. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * feat(plugins): migrate manifests to namespaced composition Move repository composition metadata under com.github.awesome-copilot, materialize reusable extensions into the plugin extensions directory, and improve contributor and PR validation guidance. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * fix(validation): address plugin review findings Restore executable build scripts, validate namespaced manifests and hook directories, improve README item counts, and manage validation comments across reruns. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76 * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Copilot-Session: 8f3a88cb-e01e-4760-8125-460490dc1a76
2026-08-08 10:09:37 +00:00 · 2026-08-06 22:29:31 +10:00
parent 6c7fe1b15f
commit a7fdcd5006
147 changed files with 2454 additions and 1320 deletions
@@ -93,7 +93,7 @@ jobs:
                  if (files.length > 0) {
                    errors.push(
                      `${pluginPath}/${subdir}/ contains ${files.length} file(s): ${files.join(', ')}. ` +
-                      `Plugin directories on main should only contain .github/plugin/plugin.json and README.md. ` +
+                      `Plugin directories on main should only contain plugin.json and README.md. ` +
                      `Agent, command, and skill files are materialized automatically during publish to marketplace.`
                    );
                  }
@@ -121,7 +121,7 @@ jobs:
                '⚠️ **Materialized files or symlinks detected in plugin directories**',
                '',
                'Plugin directories on the `main` branch should only contain:',
-                '- `.github/plugin/plugin.json` (metadata)',
+                '- `plugin.json` (metadata)',
                '- `README.md`',
                '',
                'Agent, command, and skill files are copied in automatically when publishing to `marketplace`.',
@@ -750,6 +750,7 @@ jobs:
              if (process.env.QUALITY_JOB_RESULT === 'failure' || process.env.QUALITY_JOB_RESULT === 'cancelled') {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'warning',
                  vally_lint_status: 'infra_error',
                  smoke_status: 'infra_error',
                  version_match_status: 'infra_error',
@@ -762,6 +763,7 @@ jobs:
              } else {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'warning',
                  vally_lint_status: 'infra_error',
                  smoke_status: 'infra_error',
                  version_match_status: 'infra_error',
@@ -113,6 +113,7 @@ jobs:
              if (process.env.QUALITY_JOB_RESULT === 'failure' || process.env.QUALITY_JOB_RESULT === 'cancelled') {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'warning',
                  vally_lint_status: 'infra_error',
                  smoke_status: 'infra_error',
                  version_match_status: 'infra_error',
@@ -125,6 +126,7 @@ jobs:
              } else {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'warning',
                  vally_lint_status: 'infra_error',
                  smoke_status: 'infra_error',
                  version_match_status: 'infra_error',
@@ -147,6 +147,7 @@ jobs:

            let qualityResult = {
              overall_status: 'not_run',
+              spec_compliance_status: 'not_run',
              failure_class: 'none',
              checked_plugins: [],
              summary: 'No changed external plugin entries were detected in this PR.',
@@ -155,6 +156,7 @@ jobs:
            if (detectJobResult === 'failure' || detectJobResult === 'cancelled') {
              qualityResult = {
                overall_status: 'infra_error',
+                spec_compliance_status: 'not_run',
                failure_class: 'infra',
                checked_plugins: [],
                version_match_status: 'infra_error',
@@ -165,6 +167,7 @@ jobs:
              if (qualityJobResult === 'failure' || qualityJobResult === 'cancelled') {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'not_run',
                  failure_class: 'infra',
                  checked_plugins: [],
                  version_match_status: 'infra_error',
@@ -176,6 +179,7 @@ jobs:
              } else {
                qualityResult = {
                  overall_status: 'infra_error',
+                  spec_compliance_status: 'not_run',
                  failure_class: 'infra',
                  checked_plugins: [],
                  version_match_status: 'infra_error',
@@ -201,11 +205,29 @@ jobs:
            });

            const checkedPlugins = Array.isArray(qualityResult.checked_plugins) ? qualityResult.checked_plugins : [];
+            const hasSpecWarnings = checkedPlugins.some((entry) => String(entry?.quality?.spec_compliance_status || '') === 'warning');
            const header = qualityResult.failure_class === 'submitter_fixes'
-              ? '## ⚠️ External plugin PR checks require submitter fixes'
-              : qualityResult.overall_status === 'pass' || !shouldRun
-                ? '## ✅ External plugin PR checks passed'
-                : '## ⚠️ External plugin PR checks need maintainer follow-up';
+              ? '## 🛑 External plugin PR checks failed (submitter fixes required)'
+              : qualityResult.overall_status === 'infra_error'
+                ? '## 🛑 External plugin PR checks failed (maintainer follow-up)'
+                : hasSpecWarnings
+                  ? '## ⚠️ External plugin PR checks passed with spec warnings'
+                  : qualityResult.overall_status === 'pass' || !shouldRun
+                    ? '## ✅ External plugin PR checks passed'
+                    : '## ⚠️ External plugin PR checks need maintainer follow-up';
+            const formatStatus = (rawStatus, gateName) => {
+              const status = String(rawStatus || 'not_run');
+              if (status === 'pass') {
+                return '✅ pass';
+              }
+              if (status === 'warning' || (gateName === 'spec compliance' && status === 'fail')) {
+                return '⚠️ warning';
+              }
+              if (status === 'fail' || status === 'infra_error') {
+                return '🛑 fail';
+              }
+              return '⚪ not_run';
+            };
            const MAX_GATE_OUTPUT_CHARS = 2000;
            const escapeHtml = (value) =>
              String(value || '')
@@ -273,21 +295,16 @@ jobs:
              ? checkedPlugins.map((entry) => {
                  const name = escapeMarkdownTableCell(entry?.name || 'unknown');
                  const quality = entry?.quality || {};
-                  const sourceUrl = normalizeGitHubUrl(entry?.source_tree_url || '');
+                  const sourceUrl = normalizeGitHubUrl(entry?.source_tree_url);
                  const locator = escapeMarkdownTableCell(entry?.source?.sha || entry?.source?.ref || 'repository');
-                  const sourceCell = sourceUrl ? `[${locator}](<${sourceUrl}>)` : locator;
-                  const vallyLintStatus = escapeMarkdownTableCell(quality.vally_lint_status || 'not_run');
-                  const smokeStatus = escapeMarkdownTableCell(quality.smoke_status || 'not_run');
-                  const versionMatchStatus = escapeMarkdownTableCell(quality.version_match_status || 'not_run');
-                  const refShaConsistencyStatus = escapeMarkdownTableCell(quality.ref_sha_consistency_status || 'not_run');
-                  const canvasStructureStatus = escapeMarkdownTableCell(quality.canvas_structure_status || 'not_run');
-                  const overallStatus = escapeMarkdownTableCell(quality.overall_status || 'not_run');
-                  return `| ${name} | ${vallyLintStatus} | ${smokeStatus} | ${versionMatchStatus} | ${refShaConsistencyStatus} | ${canvasStructureStatus} | ${overallStatus} | ${sourceCell} |`;
+                  const sourceCell = sourceUrl ? `[${locator}](${sourceUrl})` : locator;
+                  return `| ${name} | ${formatStatus(quality.spec_compliance_status, 'spec compliance')} | ${formatStatus(quality.vally_lint_status, 'vally lint')} | ${formatStatus(quality.smoke_status, 'install smoke test')} | ${formatStatus(quality.version_match_status, 'version match')} | ${formatStatus(quality.ref_sha_consistency_status, 'ref/sha consistency')} | ${formatStatus(quality.canvas_structure_status, 'canvas structure')} | ${formatStatus(quality.overall_status, 'overall')} | ${sourceCell} |`;
                })
-              : ['| _none_ | not_run | not_run | not_run | not_run | not_run | not_run | _n/a_ |'];
+              : ['| _none_ | ⚪ not_run | ⚪ not_run | ⚪ not_run | ⚪ not_run | ⚪ not_run | ⚪ not_run | ⚪ not_run | _n/a_ |'];
            const failureDetails = checkedPlugins.flatMap((entry) => {
              const name = String(entry?.name || 'unknown');
              const quality = entry?.quality || {};
+              const shouldShowSpec = quality.spec_compliance_status === 'warning' || String(quality.spec_compliance_output || '').trim().length > 0;
              const shouldShowVally = quality.vally_lint_status === 'fail' || quality.vally_lint_status === 'infra_error' || String(quality.vally_lint_output || '').trim().length > 0;
              const shouldShowSmoke = quality.smoke_status === 'fail' || quality.smoke_status === 'infra_error' || String(quality.smoke_output || '').trim().length > 0;
              const shouldShowVersionMatch = quality.version_match_status === 'fail' || quality.version_match_status === 'infra_error' || String(quality.version_match_output || '').trim().length > 0;
@@ -295,11 +312,14 @@ jobs:
              const shouldShowCanvasStructure = quality.canvas_structure_status === 'fail' || quality.canvas_structure_status === 'infra_error' || String(quality.canvas_structure_output || '').trim().length > 0;

              const details = [];
+              if (shouldShowSpec) {
+                details.push(formatGateOutput(name, 'spec compliance', formatStatus(quality.spec_compliance_status, 'spec compliance'), quality.spec_compliance_output));
+              }
              if (shouldShowVally) {
-                details.push(formatGateOutput(name, 'vally lint', quality.vally_lint_status, quality.vally_lint_output));
+                details.push(formatGateOutput(name, 'vally lint', formatStatus(quality.vally_lint_status, 'vally lint'), quality.vally_lint_output));
              }
              if (shouldShowSmoke) {
-                details.push(formatGateOutput(name, 'install smoke test', quality.smoke_status, quality.smoke_output));
+                details.push(formatGateOutput(name, 'install smoke test', formatStatus(quality.smoke_status, 'install smoke test'), quality.smoke_output));
              }
              if (shouldShowVersionMatch) {
                details.push(formatGateOutput(name, 'version match', quality.version_match_status, quality.version_match_output));
@@ -319,11 +339,12 @@ jobs:
              '',
              `- **Changed entries detected:** ${changedCount}`,
              `- **Workflow state label:** \`${stateLabel}\``,
+              '- **Status legend:** ✅ pass · ⚠️ warning · 🛑 fail',
              '',
              '### Per-plugin quality summary',
              '',
-              '| Plugin | vally lint | install smoke test | version match | ref/sha consistency | canvas structure | overall | source tree |',
-              '|---|---|---|---|---|---|---|---|',
+              '| Plugin | spec compliance (non-blocking) | vally lint | install smoke test | version match | ref/sha consistency | canvas structure | overall | source tree |',
+              '|---|---|---|---|---|---|---|---|---|',
              ...rows,
              '',
              ...(failureDetails.length > 0
@@ -0,0 +1,95 @@
+name: Validate Plugins and Extensions
+
+on:
+  pull_request:
+    branches: [main]
+    paths:
+      - "plugins/**"
+      - "extensions/**"
+      - "eng/validate-plugins.mjs"
+      - ".github/workflows/validate-plugins.yml"
+
+permissions:
+  contents: read
+  pull-requests: write
+
+jobs:
+  validate:
+    name: Plugin & extension spec validation
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
+
+      - name: Setup Node.js
+        uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
+        with:
+          node-version: "22"
+          cache: "npm"
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Validate plugins and extensions
+        id: validate
+        run: npm run plugin:validate
+
+      - name: Manage validation summary on PR
+        if: always()
+        uses: actions/github-script@f28e40c7f34bde8b3046d885e986cb6290c5673b # v7.1.0
+        with:
+          script: |
+            const marker = '<!-- plugin-extension-validation -->';
+            const validationOutcome = '${{ steps.validate.outcome }}';
+            const validationFailed = validationOutcome === 'failure';
+            const validationPassed = validationOutcome === 'success';
+            const body = [
+              marker,
+              '🛑 **Plugin/extension validation failed**',
+              '',
+              'One or more plugins or extensions in this PR do not pass spec validation.',
+              '',
+              'All internal plugins and extensions must include:',
+              '- `"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json"` in `plugin.json`',
+              '- A valid `name`, `description`, and `version`',
+              '- Repository composition (`agents`, `commands`, `hooks`, `mcpServers`, `skills`, and reusable `extensions`) under `extensions.com.github.awesome-copilot`',
+              '- For **extensions**: `extensions.com.github.copilot.logo` must be set to `"assets/preview.png"`',
+              '',
+              'Do not put repository composition fields at the manifest top level or directly under `extensions`; they must be nested under `extensions.com.github.awesome-copilot`.',
+              '',
+              'Run `npm run plugin:validate` locally to see the full list of errors. See `.github/skills/create-canvas-extension/SKILL.md` and `CONTRIBUTING.md` for examples.',
+            ].join('\n');
+
+            const { data: comments } = await github.rest.issues.listComments({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number,
+              per_page: 100,
+            });
+
+            const existing = comments.find((comment) => comment.body.includes(marker));
+
+            if (validationFailed && existing) {
+              await github.rest.issues.updateComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                comment_id: existing.id,
+                body,
+              });
+              console.log(`Updated existing validation comment ${existing.id}`);
+            } else if (validationFailed) {
+              await github.rest.issues.createComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                issue_number: context.issue.number,
+                body,
+              });
+              console.log('Created validation comment');
+            } else if (validationPassed && existing) {
+              await github.rest.issues.deleteComment({
+                owner: context.repo.owner,
+                repo: context.repo.repo,
+                comment_id: existing.id,
+              });
+              console.log(`Deleted stale validation comment ${existing.id}`);
+            }