mirror of
https://github.com/github/awesome-copilot.git
synced 2026-06-17 21:21:20 +00:00
Andrea Griffiths
91 lines
4.0 KiB
Markdown
91 lines
4.0 KiB
Markdown
# Chromium Control Canvas
|
|
|
|
A GitHub Copilot canvas that drives a real **headful Chromium** window via Playwright.
|
|
The host app's built-in `browser` canvas is WebKit (WKWebView); this gives you actual
|
|
Chromium, controllable both from the panel UI and by the agent.
|
|
|
|
The canvas panel is a control strip (URL bar, back/forward/reload, screenshot). A separate
|
|
Chromium window does the real rendering, because you can't embed Chromium inside a WebKit
|
|
iframe.
|
|
|
|
## Files
|
|
|
|
- `extension.mjs` — the extension: canvas declaration, Playwright launch, a loopback HTTP
|
|
server for the panel, and the agent actions.
|
|
- `index.html` — the control strip UI the panel renders.
|
|
- `package.json` — declares the `playwright` dependency and `"type": "module"`.
|
|
- `copilot-extension.json` — name/version metadata.
|
|
|
|
## Prerequisites
|
|
|
|
- **Node.js 20.19 or newer** (the Copilot SDK requires `node ^20.19.0 || >=22.12.0`).
|
|
The extension runs as a Node child process.
|
|
- The app's **canvas / UI-extensions experiment enabled**. Without it, the extension
|
|
loads but the canvas never appears in the panel. Enable it in the app's
|
|
Settings → Experiments. (This may not be available to all accounts.)
|
|
|
|
## Install
|
|
|
|
Drop this folder at `~/.copilot/extensions/chromium-control-canvas/` (user scope) or in a repo's
|
|
`.github/extensions/chromium-control-canvas/` (project scope), then install dependencies and the
|
|
Chromium binary from inside the folder you copied:
|
|
|
|
```sh
|
|
# User scope
|
|
cd ~/.copilot/extensions/chromium-control-canvas
|
|
|
|
# Or project scope, from the repository root
|
|
cd .github/extensions/chromium-control-canvas
|
|
|
|
npm install # playwright is declared in package.json
|
|
npx playwright install chromium # downloads the browser, a few hundred MB
|
|
```
|
|
|
|
Reload extensions in the app, then open the `chromium-control-canvas` canvas.
|
|
|
|
Note: copying the extension files only places the source. It does **not** run the
|
|
commands above or enable the experiment, so those steps are still required on first
|
|
setup.
|
|
|
|
## Attach to your own Chrome
|
|
|
|
By default the canvas launches the bundled Chromium with a persistent profile. To drive
|
|
a Chrome you already have running instead, start it with a debug port and pass `cdpUrl`
|
|
when opening the canvas:
|
|
|
|
```sh
|
|
google-chrome --remote-debugging-port=9222 # then open the canvas with cdpUrl: http://localhost:9222
|
|
```
|
|
|
|
In this mode the extension connects over CDP and never launches or kills your browser;
|
|
closing the canvas just disconnects.
|
|
|
|
## Agent actions
|
|
|
|
- `navigate { url }` — go to a URL or search query (blocklist-guarded).
|
|
- `back` / `forward` / `reload` — history navigation.
|
|
- `current_url` — current URL and page title.
|
|
- `snapshot` — structured list of visible interactive elements, each with a stable ref.
|
|
- `click { ref | selector }` — click an element by snapshot ref or CSS selector.
|
|
- `type { ref | selector, text, submit? }` — fill an input; optionally press Enter.
|
|
- `screenshot { fullPage? }` — save a PNG to `artifacts/` and return its path and size.
|
|
|
|
## Notes
|
|
|
|
- A persistent profile is stored under
|
|
`$COPILOT_HOME/extensions/chromium-control-canvas/profile` (default
|
|
`~/.copilot/extensions/chromium-control-canvas/profile`) so logins survive restarts.
|
|
**Do not commit or share this folder** — it contains real session cookies.
|
|
- Raw `evaluate` (arbitrary in-page JS) is intentionally omitted.
|
|
- `navigate` is checked against a blocklist, and a request interceptor also blocks
|
|
navigations to blocked hosts that happen via in-page redirects. The shipped
|
|
`BLOCKLIST` entries are illustrative examples, not real coverage — edit the list in
|
|
`extension.mjs` to fit your environment.
|
|
- The loopback control server requires a per-launch token (templated into the panel),
|
|
so other pages in your browser can't drive it.
|
|
- Typed text (e.g. passwords) is redacted in `audit.log`, and password field values are
|
|
excluded from snapshots.
|
|
- Generated at runtime and not part of the source: `node_modules/` in the copied
|
|
extension folder, plus `profile/`, `artifacts/`, and `audit.log` under
|
|
`$COPILOT_HOME/extensions/chromium-control-canvas/`.
|