awesome-copilot/extensions/chromium-control-canvas

Chromium Control Canvas

A GitHub Copilot canvas that drives a real headful Chromium window via Playwright. The host app's built-in browser canvas is WebKit (WKWebView); this gives you actual Chromium, controllable both from the panel UI and by the agent.

The canvas panel is a control strip (URL bar, back/forward/reload, screenshot). A separate Chromium window does the real rendering, because you can't embed Chromium inside a WebKit iframe.

Files

• extension.mjs — the extension: canvas declaration, Playwright launch, a loopback HTTP server for the panel, and the agent actions.
• index.html — the control strip UI the panel renders.
• package.json — declares the playwright dependency and "type": "module".
• copilot-extension.json — name/version metadata.

Prerequisites

• Node.js 20.19 or newer (the Copilot SDK requires node ^20.19.0 || >=22.12.0). The extension runs as a Node child process.
• The app's canvas / UI-extensions experiment enabled. Without it, the extension loads but the canvas never appears in the panel. Enable it in the app's Settings → Experiments. (This may not be available to all accounts.)

Install

Drop this folder at ~/.copilot/extensions/chromium-control-canvas/ (user scope) or in a repo's .github/extensions/chromium-control-canvas/ (project scope), then install dependencies and the Chromium binary from inside the folder you copied:

# User scope
cd ~/.copilot/extensions/chromium-control-canvas

# Or project scope, from the repository root
cd .github/extensions/chromium-control-canvas

npm install                     # playwright is declared in package.json
npx playwright install chromium # downloads the browser, a few hundred MB

Reload extensions in the app, then open the chromium-control-canvas canvas.

Note: copying the extension files only places the source. It does not run the commands above or enable the experiment, so those steps are still required on first setup.

Attach to your own Chrome

By default the canvas launches the bundled Chromium with a persistent profile. To drive a Chrome you already have running instead, start it with a debug port and pass cdpUrl when opening the canvas:

google-chrome --remote-debugging-port=9222   # then open the canvas with cdpUrl: http://localhost:9222

In this mode the extension connects over CDP and never launches or kills your browser; closing the canvas just disconnects.

Agent actions

• navigate { url } — go to a URL or search query (blocklist-guarded).
• back / forward / reload — history navigation.
• current_url — current URL and page title.
• snapshot — structured list of visible interactive elements, each with a stable ref.
• click { ref | selector } — click an element by snapshot ref or CSS selector.
• type { ref | selector, text, submit? } — fill an input; optionally press Enter.
• screenshot { fullPage? } — save a PNG to artifacts/ and return its path and size.

Notes

• A persistent profile is stored under $COPILOT_HOME/extensions/chromium-control-canvas/profile (default ~/.copilot/extensions/chromium-control-canvas/profile) so logins survive restarts. Do not commit or share this folder — it contains real session cookies.
• Raw evaluate (arbitrary in-page JS) is intentionally omitted.
• navigate is checked against a blocklist, and a request interceptor also blocks navigations to blocked hosts that happen via in-page redirects. The shipped BLOCKLIST entries are illustrative examples, not real coverage — edit the list in extension.mjs to fit your environment.
• The loopback control server requires a per-launch token (templated into the panel), so other pages in your browser can't drive it.
• Typed text (e.g. passwords) is redacted in audit.log, and password field values are excluded from snapshots.
• Generated at runtime and not part of the source: node_modules/ in the copied extension folder, plus profile/, artifacts/, and audit.log under $COPILOT_HOME/extensions/chromium-control-canvas/.