Add automate-this plugin: screen recording to automation scripts (#930)

* Add automate-this plugin: screen recording to automation scripts

New plugin that analyzes screen recordings of manual processes and
proposes working automation scripts at multiple complexity tiers.

Features:
- Frame extraction via ffmpeg for visual process reconstruction
- Optional audio transcription via Whisper for narrated recordings
- Environment fingerprinting to constrain proposals to installed tools
- Application-specific automation strategies (browser, spreadsheet,
  email, terminal, file management, macOS-native)
- Three-tier proposal system (quick win, full script, scheduled automation)
- Dry-run support for safe testing of all proposals

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update skills/automate-this/SKILL.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Address code review feedback on automate-this plugin

- Fix privacy section: clarify that frames/audio are processed locally
  but sent to the Copilot model for analysis (not purely local)
- Replace 'which' with 'command -v' for reliable tool detection with
  explicit NO_FFMPEG/NO_WHISPER sentinels
- Use per-run secure temp directory (mktemp -d, mode 0700) instead of
  fixed /tmp/ paths to prevent other users reading extracted frames
- Add -y flag and -loglevel warning to ffmpeg calls to prevent
  overwrite prompts and avoid piping hiding exit codes
- Add whisper-cpp CLI invocation path so users who install only
  whisper-cpp get working transcription
- Fix grouped command syntax in environment fingerprint to prevent
  false 'not installed' fallthrough

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Dan Velton
2026-03-08 20:46:32 -07:00
committed by GitHub
parent e05c82c43f
commit 5bed97868a
6 changed files with 362 additions and 0 deletions

View File

@@ -10,6 +10,12 @@
"email": "copilot@github.com"
},
"plugins": [
{
"name": "automate-this",
"source": "automate-this",
"description": "Record your screen doing a manual process, drop the video on your Desktop, and let Copilot CLI analyze it frame-by-frame to build working automation scripts. Supports narrated recordings with audio transcription.",
"version": "1.0.0"
},
{
"name": "awesome-copilot",
"source": "awesome-copilot",