Commit Graph

1 Commits

Author SHA1 Message Date
Dan Velton
5bed97868a Add automate-this plugin: screen recording to automation scripts (#930)
* Add automate-this plugin: screen recording to automation scripts

New plugin that analyzes screen recordings of manual processes and
proposes working automation scripts at multiple complexity tiers.

Features:
- Frame extraction via ffmpeg for visual process reconstruction
- Optional audio transcription via Whisper for narrated recordings
- Environment fingerprinting to constrain proposals to installed tools
- Application-specific automation strategies (browser, spreadsheet,
  email, terminal, file management, macOS-native)
- Three-tier proposal system (quick win, full script, scheduled automation)
- Dry-run support for safe testing of all proposals

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update skills/automate-this/SKILL.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Address code review feedback on automate-this plugin

- Fix privacy section: clarify that frames/audio are processed locally
  but sent to the Copilot model for analysis (not purely local)
- Replace 'which' with 'command -v' for reliable tool detection with
  explicit NO_FFMPEG/NO_WHISPER sentinels
- Use per-run secure temp directory (mktemp -d, mode 0700) instead of
  fixed /tmp/ paths to prevent other users reading extracted frames
- Add -y flag and -loglevel warning to ffmpeg calls to prevent
  overwrite prompts and avoid piping hiding exit codes
- Add whisper-cpp CLI invocation path so users who install only
  whisper-cpp get working transcription
- Fix grouped command syntax in environment fingerprint to prevent
  false 'not installed' fallthrough

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-03-09 14:46:32 +11:00