Awesome/awesome-copilot
1
0
Fork 0
mirror of https://github.com/github/awesome-copilot.git synced 2026-03-12 12:15:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Add automate-this plugin: screen recording to automation scripts (#930)

Browse Source 
* Add automate-this plugin: screen recording to automation scripts

New plugin that analyzes screen recordings of manual processes and
proposes working automation scripts at multiple complexity tiers.

Features:
- Frame extraction via ffmpeg for visual process reconstruction
- Optional audio transcription via Whisper for narrated recordings
- Environment fingerprinting to constrain proposals to installed tools
- Application-specific automation strategies (browser, spreadsheet,
  email, terminal, file management, macOS-native)
- Three-tier proposal system (quick win, full script, scheduled automation)
- Dry-run support for safe testing of all proposals

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Update skills/automate-this/SKILL.md

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Address code review feedback on automate-this plugin

- Fix privacy section: clarify that frames/audio are processed locally
  but sent to the Copilot model for analysis (not purely local)
- Replace 'which' with 'command -v' for reliable tool detection with
  explicit NO_FFMPEG/NO_WHISPER sentinels
- Use per-run secure temp directory (mktemp -d, mode 0700) instead of
  fixed /tmp/ paths to prevent other users reading extracted frames
- Add -y flag and -loglevel warning to ffmpeg calls to prevent
  overwrite prompts and avoid piping hiding exit codes
- Add whisper-cpp CLI invocation path so users who install only
  whisper-cpp get working transcription
- Fix grouped command syntax in environment fingerprint to prevent
  false 'not installed' fallthrough

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Dan Velton
2026-03-08 20:46:32 -07:00
committed by GitHub
parent e05c82c43f
commit 5bed97868a
6 changed files with 362 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/README.plugins.md
View File

@@ -25,6 +25,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t
| Name | Description | Items | Tags |
| ---- | ----------- | ----- | ---- |
| [automate-this](../plugins/automate-this/README.md) | Record your screen doing a manual process, drop the video on your Desktop, and let Copilot CLI analyze it frame-by-frame to build working automation scripts. Supports narrated recordings with audio transcription. | 1 items | automation, screen-recording, workflow, video-analysis, process-automation, scripting, productivity, copilot-cli |
| [awesome-copilot](../plugins/awesome-copilot/README.md) | Meta prompts that help you discover and generate curated GitHub Copilot agents, instructions, prompts, and skills. | 4 items | github-copilot, discovery, meta, prompt-engineering, agents |
| [azure-cloud-development](../plugins/azure-cloud-development/README.md) | Comprehensive Azure cloud development tools including Infrastructure as Code, serverless functions, architecture patterns, and cost optimization for building scalable cloud applications. | 11 items | azure, cloud, infrastructure, bicep, terraform, serverless, architecture, devops |
| [cast-imaging](../plugins/cast-imaging/README.md) | A comprehensive collection of specialized agents for software analysis, impact assessment, structural quality advisories, and architectural review using CAST Imaging. | 3 items | cast-imaging, software-analysis, architecture, quality, impact-analysis, devops |