From 46f6f3e6db7e0b424a8565080f8844a45de81058 Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 13:51:45 -0400 Subject: [PATCH 01/16] Add Java SDK cookbook with 7 recipes Add complete Java cookbook matching the pattern of existing .NET, Go, Node.js, and Python cookbooks. All 7 recipes included: - Ralph Loop: Autonomous AI task loops with JBang - Error Handling: try-with-resources, ExecutionException, timeouts - Multiple Sessions: Parallel sessions with CompletableFuture - Managing Local Files: AI-powered file organization - PR Visualization: Interactive PR age charts - Persisting Sessions: Save/resume with custom IDs - Accessibility Report: WCAG reports via Playwright MCP Each recipe includes both markdown documentation and a standalone JBang-runnable Java file in recipe/. --- cookbook/README.md | 2 +- cookbook/copilot-sdk/README.md | 19 +- cookbook/copilot-sdk/java/README.md | 21 ++ .../copilot-sdk/java/accessibility-report.md | 238 +++++++++++++ cookbook/copilot-sdk/java/error-handling.md | 183 ++++++++++ .../copilot-sdk/java/managing-local-files.md | 209 ++++++++++++ .../copilot-sdk/java/multiple-sessions.md | 144 ++++++++ .../copilot-sdk/java/persisting-sessions.md | 320 ++++++++++++++++++ cookbook/copilot-sdk/java/pr-visualization.md | 235 +++++++++++++ cookbook/copilot-sdk/java/ralph-loop.md | 247 ++++++++++++++ .../java/recipe/AccessibilityReport.java | 123 +++++++ .../java/recipe/ErrorHandling.java | 34 ++ .../java/recipe/ManagingLocalFiles.java | 64 ++++ .../java/recipe/MultipleSessions.java | 37 ++ .../java/recipe/PRVisualization.java | 182 ++++++++++ .../java/recipe/PersistingSessions.java | 34 ++ cookbook/copilot-sdk/java/recipe/README.md | 71 ++++ .../copilot-sdk/java/recipe/RalphLoop.java | 55 +++ 18 files changed, 2216 insertions(+), 2 deletions(-) create mode 100644 cookbook/copilot-sdk/java/README.md create mode 100644 cookbook/copilot-sdk/java/accessibility-report.md create mode 100644 cookbook/copilot-sdk/java/error-handling.md create mode 100644 cookbook/copilot-sdk/java/managing-local-files.md create mode 100644 cookbook/copilot-sdk/java/multiple-sessions.md create mode 100644 cookbook/copilot-sdk/java/persisting-sessions.md create mode 100644 cookbook/copilot-sdk/java/pr-visualization.md create mode 100644 cookbook/copilot-sdk/java/ralph-loop.md create mode 100644 cookbook/copilot-sdk/java/recipe/AccessibilityReport.java create mode 100644 cookbook/copilot-sdk/java/recipe/ErrorHandling.java create mode 100644 cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java create mode 100644 cookbook/copilot-sdk/java/recipe/MultipleSessions.java create mode 100644 cookbook/copilot-sdk/java/recipe/PRVisualization.java create mode 100644 cookbook/copilot-sdk/java/recipe/PersistingSessions.java create mode 100644 cookbook/copilot-sdk/java/recipe/README.md create mode 100644 cookbook/copilot-sdk/java/recipe/RalphLoop.java diff --git a/cookbook/README.md b/cookbook/README.md index 797ce76d..0ca90d3a 100644 --- a/cookbook/README.md +++ b/cookbook/README.md @@ -10,7 +10,7 @@ The cookbook is organized by tool or product, with recipes collected by language Ready-to-use recipes for building with the GitHub Copilot SDK across multiple languages. -- **[Copilot SDK Cookbook](copilot-sdk/)** - Recipes for .NET, Go, Node.js, and Python +- **[Copilot SDK Cookbook](copilot-sdk/)** - Recipes for .NET, Go, Java, Node.js, and Python - Error handling, session management, file operations, and more - Runnable examples for each language - Best practices and complete implementation guides diff --git a/cookbook/copilot-sdk/README.md b/cookbook/copilot-sdk/README.md index 3e2738d1..d9a127f9 100644 --- a/cookbook/copilot-sdk/README.md +++ b/cookbook/copilot-sdk/README.md @@ -44,6 +44,16 @@ This cookbook collects small, focused recipes showing how to accomplish common t - [Persisting Sessions](go/persisting-sessions.md): Save and resume sessions across restarts. - [Accessibility Report](go/accessibility-report.md): Generate WCAG accessibility reports using the Playwright MCP server. +### Java + +- [Ralph Loop](java/ralph-loop.md): Build autonomous AI coding loops with fresh context per iteration, planning/building modes, and backpressure. +- [Error Handling](java/error-handling.md): Handle errors gracefully including connection failures, timeouts, and cleanup. +- [Multiple Sessions](java/multiple-sessions.md): Manage multiple independent conversations simultaneously. +- [Managing Local Files](java/managing-local-files.md): Organize files by metadata using AI-powered grouping strategies. +- [PR Visualization](java/pr-visualization.md): Generate interactive PR age charts using GitHub MCP Server. +- [Persisting Sessions](java/persisting-sessions.md): Save and resume sessions across restarts. +- [Accessibility Report](java/accessibility-report.md): Generate WCAG accessibility reports using the Playwright MCP server. + ## How to Use - Browse your language section above and open the recipe links @@ -84,6 +94,13 @@ cd go/cookbook/recipe go run .go ``` +### Java + +```bash +cd java/cookbook/recipe +jbang .java +``` + ## Contributing - Propose or add a new recipe by creating a markdown file in your language's `cookbook/` folder and a runnable example in `recipe/` @@ -91,4 +108,4 @@ go run .go ## Status -Cookbook structure is complete with 7 recipes across all 4 supported languages. Each recipe includes both markdown documentation and runnable examples. +Cookbook structure is complete with 7 recipes across all 5 supported languages. Each recipe includes both markdown documentation and runnable examples. diff --git a/cookbook/copilot-sdk/java/README.md b/cookbook/copilot-sdk/java/README.md new file mode 100644 index 00000000..fb335a19 --- /dev/null +++ b/cookbook/copilot-sdk/java/README.md @@ -0,0 +1,21 @@ +# GitHub Copilot SDK Cookbook β€” Java + +This folder hosts short, practical recipes for using the GitHub Copilot SDK with Java. Each recipe is concise, copy‑pasteable, and points to fuller examples and tests. All examples can be run directly with [JBang](https://www.jbang.dev/). + +## Recipes + +- [Ralph Loop](ralph-loop.md): Build autonomous AI coding loops with fresh context per iteration, planning/building modes, and backpressure. +- [Error Handling](error-handling.md): Handle errors gracefully including connection failures, timeouts, and cleanup. +- [Multiple Sessions](multiple-sessions.md): Manage multiple independent conversations simultaneously. +- [Managing Local Files](managing-local-files.md): Organize files by metadata using AI-powered grouping strategies. +- [PR Visualization](pr-visualization.md): Generate interactive PR age charts using GitHub MCP Server. +- [Persisting Sessions](persisting-sessions.md): Save and resume sessions across restarts. +- [Accessibility Report](accessibility-report.md): Generate WCAG accessibility reports using the Playwright MCP server. + +## Contributing + +Add a new recipe by creating a markdown file in this folder and linking it above. Follow repository guidance in [CONTRIBUTING.md](../../../CONTRIBUTING.md). + +## Status + +These recipes are complete, practical examples and can be used directly or adapted for your own projects. diff --git a/cookbook/copilot-sdk/java/accessibility-report.md b/cookbook/copilot-sdk/java/accessibility-report.md new file mode 100644 index 00000000..2b6abbe6 --- /dev/null +++ b/cookbook/copilot-sdk/java/accessibility-report.md @@ -0,0 +1,238 @@ +# Generating Accessibility Reports + +Build a CLI tool that analyzes web page accessibility using the Playwright MCP server and generates detailed WCAG-compliant reports with optional test generation. + +> **Runnable example:** [recipe/AccessibilityReport.java](recipe/AccessibilityReport.java) +> +> ```bash +> jbang recipe/AccessibilityReport.java +> ``` + +## Example scenario + +You want to audit a website's accessibility compliance. This tool navigates to a URL using Playwright, captures an accessibility snapshot, and produces a structured report covering WCAG criteria like landmarks, heading hierarchy, focus management, and touch targets. It can also generate Playwright test files to automate future accessibility checks. + +## Prerequisites + +Install [JBang](https://www.jbang.dev/) and ensure `npx` is available (Node.js installed) for the Playwright MCP server: + +```bash +# macOS (using Homebrew) +brew install jbangdev/tap/jbang + +# Verify npx is available (needed for Playwright MCP) +npx --version +``` + +## Usage + +```bash +jbang AccessibilityReport.java +# Enter a URL when prompted +``` + +## Full example: AccessibilityReport.java + +```java +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.io.*; +import java.util.*; +import java.util.concurrent.*; + +public class AccessibilityReport { + public static void main(String[] args) throws Exception { + System.out.println("=== Accessibility Report Generator ===\n"); + + var reader = new BufferedReader(new InputStreamReader(System.in)); + + System.out.print("Enter URL to analyze: "); + String url = reader.readLine().trim(); + if (url.isEmpty()) { + System.out.println("No URL provided. Exiting."); + return; + } + if (!url.startsWith("http://") && !url.startsWith("https://")) { + url = "https://" + url; + } + + System.out.printf("%nAnalyzing: %s%n", url); + System.out.println("Please wait...\n"); + + try (var client = new CopilotClient()) { + client.start().get(); + + // Configure Playwright MCP server for browser automation + var mcpConfig = new McpServerConfig() + .setType("local") + .setCommand("npx") + .setArgs(List.of("@playwright/mcp@latest")) + .setTools(List.of("*")); + + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("claude-opus-4.6") + .setStreaming(true) + .setMcpServers(Map.of("playwright", mcpConfig)) + ).get(); + + // Stream output token-by-token + var idleLatch = new CountDownLatch(1); + + session.on(AssistantMessageDeltaEvent.class, + ev -> System.out.print(ev.getData().deltaContent())); + + session.on(SessionIdleEvent.class, + ev -> idleLatch.countDown()); + + session.on(SessionErrorEvent.class, ev -> { + System.err.printf("%nError: %s%n", ev.getData().message()); + idleLatch.countDown(); + }); + + String prompt = """ + Use the Playwright MCP server to analyze the accessibility of this webpage: %s + + Please: + 1. Navigate to the URL using playwright-browser_navigate + 2. Take an accessibility snapshot using playwright-browser_snapshot + 3. Analyze the snapshot and provide a detailed accessibility report + + Format the report with emoji indicators: + - πŸ“Š Accessibility Report header + - βœ… What's Working Well (table with Category, Status, Details) + - ⚠️ Issues Found (table with Severity, Issue, WCAG Criterion, Recommendation) + - πŸ“‹ Stats Summary (links, headings, focusable elements, landmarks) + - βš™οΈ Priority Recommendations + + Use βœ… for pass, πŸ”΄ for high severity issues, 🟑 for medium severity, ❌ for missing items. + Include actual findings from the page analysis. + """.formatted(url); + + session.send(new MessageOptions().setPrompt(prompt)); + idleLatch.await(); + + System.out.println("\n\n=== Report Complete ===\n"); + + // Prompt user for test generation + System.out.print("Would you like to generate Playwright accessibility tests? (y/n): "); + String generateTests = reader.readLine().trim(); + + if (generateTests.equalsIgnoreCase("y") || generateTests.equalsIgnoreCase("yes")) { + var testLatch = new CountDownLatch(1); + + session.on(SessionIdleEvent.class, + ev -> testLatch.countDown()); + + String testPrompt = """ + Based on the accessibility report you just generated for %s, + create Playwright accessibility tests in Java. + + Include tests for: lang attribute, title, heading hierarchy, alt text, + landmarks, skip navigation, focus indicators, and touch targets. + Use Playwright's accessibility testing features with helpful comments. + Output the complete test file. + """.formatted(url); + + System.out.println("\nGenerating accessibility tests...\n"); + session.send(new MessageOptions().setPrompt(testPrompt)); + testLatch.await(); + + System.out.println("\n\n=== Tests Generated ==="); + } + + session.close(); + } + } +} +``` + +## How it works + +1. **Playwright MCP server**: Configures a local MCP server running `@playwright/mcp` to provide browser automation tools +2. **Streaming output**: Uses `streaming: true` and `AssistantMessageDeltaEvent` for real-time token-by-token output +3. **Accessibility snapshot**: Playwright's `browser_snapshot` tool captures the full accessibility tree of the page +4. **Structured report**: The prompt engineers a consistent WCAG-aligned report format with emoji severity indicators +5. **Test generation**: Optionally generates Playwright accessibility tests based on the analysis + +## Key concepts + +### MCP server configuration + +The recipe configures a local MCP server that runs alongside the session: + +```java +var mcpConfig = new McpServerConfig() + .setType("local") + .setCommand("npx") + .setArgs(List.of("@playwright/mcp@latest")) + .setTools(List.of("*")); + +var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setMcpServers(Map.of("playwright", mcpConfig)) +).get(); +``` + +This gives the model access to Playwright browser tools like `browser_navigate`, `browser_snapshot`, and `browser_click`. + +### Streaming with events + +Unlike `sendAndWait`, this recipe uses streaming for real-time output: + +```java +session.on(AssistantMessageDeltaEvent.class, + ev -> System.out.print(ev.getData().deltaContent())); + +session.on(SessionIdleEvent.class, + ev -> idleLatch.countDown()); +``` + +A `CountDownLatch` synchronizes the main thread with the async event stream β€” when the session becomes idle, the latch releases and the program continues. + +## Sample interaction + +``` +=== Accessibility Report Generator === + +Enter URL to analyze: github.com + +Analyzing: https://github.com +Please wait... + +πŸ“Š Accessibility Report: GitHub (github.com) + +βœ… What's Working Well +| Category | Status | Details | +|----------|--------|---------| +| Language | βœ… Pass | lang="en" properly set | +| Page Title | βœ… Pass | "GitHub" is recognizable | +| Heading Hierarchy | βœ… Pass | Proper H1/H2 structure | +| Images | βœ… Pass | All images have alt text | + +⚠️ Issues Found +| Severity | Issue | WCAG Criterion | Recommendation | +|----------|-------|----------------|----------------| +| 🟑 Medium | Some links lack descriptive text | 2.4.4 | Add aria-label to icon-only links | + +πŸ“‹ Stats Summary +- Total Links: 47 +- Total Headings: 8 (1Γ— H1, proper hierarchy) +- Focusable Elements: 52 +- Landmarks Found: banner βœ…, navigation βœ…, main βœ…, footer βœ… + +=== Report Complete === + +Would you like to generate Playwright accessibility tests? (y/n): y + +Generating accessibility tests... +[Generated test file output...] + +=== Tests Generated === +``` diff --git a/cookbook/copilot-sdk/java/error-handling.md b/cookbook/copilot-sdk/java/error-handling.md new file mode 100644 index 00000000..8c2dd026 --- /dev/null +++ b/cookbook/copilot-sdk/java/error-handling.md @@ -0,0 +1,183 @@ +# Error Handling Patterns + +Handle errors gracefully in your Copilot SDK applications. + +> **Runnable example:** [recipe/ErrorHandling.java](recipe/ErrorHandling.java) +> +> ```bash +> jbang recipe/ErrorHandling.java +> ``` + +## Example scenario + +You need to handle various error conditions like connection failures, timeouts, and invalid responses. + +## Basic try-with-resources + +Java's `try-with-resources` ensures the client is always cleaned up, even when exceptions occur. + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.json.*; + +public class BasicErrorHandling { + public static void main(String[] args) { + try (var client = new CopilotClient()) { + client.start().get(); + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5")).get(); + + var response = session.sendAndWait( + new MessageOptions().setPrompt("Hello!")).get(); + System.out.println(response.getData().content()); + + session.close(); + } catch (Exception ex) { + System.err.println("Error: " + ex.getMessage()); + } + } +} +``` + +## Handling specific error types + +Every `CompletableFuture.get()` call wraps failures in `ExecutionException`. Unwrap the cause to inspect the real error. + +```java +import java.io.IOException; +import java.util.concurrent.ExecutionException; + +try (var client = new CopilotClient()) { + client.start().get(); +} catch (ExecutionException ex) { + var cause = ex.getCause(); + if (cause instanceof IOException) { + System.err.println("Copilot CLI not found or could not connect: " + cause.getMessage()); + } else { + System.err.println("Unexpected error: " + cause.getMessage()); + } +} catch (InterruptedException ex) { + Thread.currentThread().interrupt(); + System.err.println("Interrupted while starting client."); +} +``` + +## Timeout handling + +Use the overloaded `get(timeout, unit)` on `CompletableFuture` to enforce time limits. + +```java +import java.util.concurrent.TimeUnit; +import java.util.concurrent.TimeoutException; + +var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5")).get(); + +try { + var response = session.sendAndWait( + new MessageOptions().setPrompt("Complex question...")) + .get(30, TimeUnit.SECONDS); + + System.out.println(response.getData().content()); +} catch (TimeoutException ex) { + System.err.println("Request timed out after 30 seconds."); + session.abort().get(); +} +``` + +## Aborting a request + +Cancel an in-flight request by calling `session.abort()`. + +```java +var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5")).get(); + +// Start a request without waiting +session.send(new MessageOptions().setPrompt("Write a very long story...")); + +// Abort after some condition +Thread.sleep(5000); +session.abort().get(); +System.out.println("Request aborted."); +``` + +## Graceful shutdown + +Use a JVM shutdown hook to clean up when the process is interrupted. + +```java +var client = new CopilotClient(); +client.start().get(); + +Runtime.getRuntime().addShutdownHook(new Thread(() -> { + System.out.println("Shutting down..."); + try { + client.close(); + } catch (Exception ex) { + System.err.println("Cleanup error: " + ex.getMessage()); + } +})); +``` + +## Try-with-resources (nested) + +When working with multiple sessions, nest `try-with-resources` blocks to guarantee each resource is closed. + +```java +try (var client = new CopilotClient()) { + client.start().get(); + + try (var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5")).get()) { + + session.sendAndWait( + new MessageOptions().setPrompt("Hello!")).get(); + } // session is closed here + +} // client is closed here +``` + +## Handling tool errors + +When defining tools, return an error result to signal a failure back to the model instead of throwing. + +```java +import com.github.copilot.sdk.json.ToolResultObject; + +session.addTool( + ToolDefinition.builder() + .name("read_file") + .description("Read a file from disk") + .parameter("path", "string", "File path", true) + .build(), + (args) -> { + try { + var content = java.nio.file.Files.readString( + java.nio.file.Path.of(args.get("path").toString())); + return ToolResultObject.success(content); + } catch (IOException ex) { + return ToolResultObject.error("Failed to read file: " + ex.getMessage()); + } + } +); +``` + +## Best practices + +1. **Use try-with-resources**: Always wrap `CopilotClient` (and sessions, if `AutoCloseable`) in try-with-resources to guarantee cleanup. +2. **Unwrap `ExecutionException`**: Call `getCause()` to inspect the real error β€” the outer `ExecutionException` is just a `CompletableFuture` wrapper. +3. **Restore interrupt flag**: When catching `InterruptedException`, call `Thread.currentThread().interrupt()` to preserve the interrupted status. +4. **Set timeouts**: Use `get(timeout, TimeUnit)` instead of bare `get()` for any call that could block indefinitely. +5. **Return tool errors, don't throw**: Use `ToolResultObject.error()` so the model can recover gracefully. +6. **Log errors**: Capture error details for debugging β€” consider a logging framework like SLF4J for production applications. diff --git a/cookbook/copilot-sdk/java/managing-local-files.md b/cookbook/copilot-sdk/java/managing-local-files.md new file mode 100644 index 00000000..d347c7fe --- /dev/null +++ b/cookbook/copilot-sdk/java/managing-local-files.md @@ -0,0 +1,209 @@ +# Grouping Files by Metadata + +Use Copilot to intelligently organize files in a folder based on their metadata. + +> **Runnable example:** [recipe/ManagingLocalFiles.java](recipe/ManagingLocalFiles.java) +> +> ```bash +> jbang recipe/ManagingLocalFiles.java +> ``` + +## Example scenario + +You have a folder with many files and want to organize them into subfolders based on metadata like file type, creation date, size, or other attributes. Copilot can analyze the files and suggest or execute a grouping strategy. + +## Example code + +**Usage:** +```bash +# Use with a specific folder (recommended) +jbang recipe/ManagingLocalFiles.java /path/to/your/folder + +# Or run without arguments to use a safe default (temp directory) +jbang recipe/ManagingLocalFiles.java +``` + +**Code:** +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.events.SessionIdleEvent; +import com.github.copilot.sdk.events.ToolExecutionCompleteEvent; +import com.github.copilot.sdk.events.ToolExecutionStartEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; +import java.nio.file.Paths; +import java.util.concurrent.CountDownLatch; + +public class ManagingLocalFiles { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Create session + var session = client.createSession( + new SessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setModel("gpt-5")).get(); + + // Set up event handlers + var done = new CountDownLatch(1); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nCopilot: " + msg.getData().content()) + ); + + session.on(ToolExecutionStartEvent.class, evt -> + System.out.println(" β†’ Running: " + evt.getData().toolName()) + ); + + session.on(ToolExecutionCompleteEvent.class, evt -> + System.out.println(" βœ“ Completed: " + evt.getData().toolCallId()) + ); + + session.on(SessionIdleEvent.class, evt -> done.countDown()); + + // Ask Copilot to organize files - using a safe example folder + // For real use, replace with your target folder + String targetFolder = args.length > 0 ? args[0] : + System.getProperty("java.io.tmpdir") + "/example-files"; + + String prompt = String.format(""" + Analyze the files in "%s" and show how you would organize them into subfolders. + + 1. First, list all files and their metadata + 2. Preview grouping by file extension + 3. Suggest appropriate subfolders (e.g., "images", "documents", "videos") + + IMPORTANT: DO NOT move any files. Only show the plan. + """, targetFolder); + + session.send(new MessageOptions().setPrompt(prompt)); + + // Wait for completion + done.await(); + + session.close(); + } + } +} +``` + +## Grouping strategies + +### By file extension + +```java +// Groups files like: +// images/ -> .jpg, .png, .gif +// documents/ -> .pdf, .docx, .txt +// videos/ -> .mp4, .avi, .mov +``` + +### By creation date + +```java +// Groups files like: +// 2024-01/ -> files created in January 2024 +// 2024-02/ -> files created in February 2024 +``` + +### By file size + +```java +// Groups files like: +// tiny-under-1kb/ +// small-under-1mb/ +// medium-under-100mb/ +// large-over-100mb/ +``` + +## Dry-run mode + +For safety, you can ask Copilot to only preview changes: + +```java +String prompt = String.format(""" + Analyze files in "%s" and show me how you would organize them + by file type. DO NOT move any files - just show me the plan. + """, targetFolder); + +session.send(new MessageOptions().setPrompt(prompt)); +``` + +## Custom grouping with AI analysis + +Let Copilot determine the best grouping based on file content: + +```java +String prompt = String.format(""" + Look at the files in "%s" and suggest a logical organization. + Consider: + - File names and what they might contain + - File types and their typical uses + - Date patterns that might indicate projects or events + + Propose folder names that are descriptive and useful. + """, targetFolder); + +session.send(new MessageOptions().setPrompt(prompt)); +``` + +## Interactive file organization + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; +import java.io.BufferedReader; +import java.io.InputStreamReader; + +public class InteractiveFileOrganizer { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient(); + var reader = new BufferedReader(new InputStreamReader(System.in))) { + + client.start().get(); + + var session = client.createSession( + new SessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setModel("gpt-5")).get(); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nCopilot: " + msg.getData().content()) + ); + + System.out.print("Enter folder path to organize: "); + String folderPath = reader.readLine(); + + String initialPrompt = String.format(""" + Analyze the files in "%s" and suggest an organization strategy. + Wait for my confirmation before making any changes. + """, folderPath); + + session.send(new MessageOptions().setPrompt(initialPrompt)); + + // Interactive loop + System.out.println("\nEnter commands (or 'exit' to quit):"); + String line; + while ((line = reader.readLine()) != null) { + if (line.equalsIgnoreCase("exit")) { + break; + } + session.send(new MessageOptions().setPrompt(line)); + } + + session.close(); + } + } +} +``` + +## Safety considerations + +1. **Confirm before moving**: Ask Copilot to confirm before executing moves +2. **Handle duplicates**: Consider what happens if a file with the same name exists +3. **Preserve originals**: Consider copying instead of moving for important files +4. **Test with dry-run**: Always test with a dry-run first to preview the changes diff --git a/cookbook/copilot-sdk/java/multiple-sessions.md b/cookbook/copilot-sdk/java/multiple-sessions.md new file mode 100644 index 00000000..585ee5fd --- /dev/null +++ b/cookbook/copilot-sdk/java/multiple-sessions.md @@ -0,0 +1,144 @@ +# Working with Multiple Sessions + +Manage multiple independent conversations simultaneously. + +> **Runnable example:** [recipe/MultipleSessions.java](recipe/MultipleSessions.java) +> +> ```bash +> jbang MultipleSessions.java +> ``` + +## Example scenario + +You need to run multiple conversations in parallel, each with its own context and history. + +## Java + +```java +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.json.*; + +var client = new CopilotClient(); +client.start().get(); + +// Create multiple independent sessions +var session1 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); +var session2 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); +var session3 = client.createSession(new SessionConfig() + .setModel("claude-sonnet-4.5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + +// Each session maintains its own conversation history +session1.sendAndWait(new MessageOptions().setPrompt("You are helping with a Python project")).get(); +session2.sendAndWait(new MessageOptions().setPrompt("You are helping with a TypeScript project")).get(); +session3.sendAndWait(new MessageOptions().setPrompt("You are helping with a Go project")).get(); + +// Follow-up messages stay in their respective contexts +session1.sendAndWait(new MessageOptions().setPrompt("How do I create a virtual environment?")).get(); +session2.sendAndWait(new MessageOptions().setPrompt("How do I set up tsconfig?")).get(); +session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); + +// Clean up all sessions +session1.destroy().get(); +session2.destroy().get(); +session3.destroy().get(); +client.stop().get(); +``` + +## Custom session IDs + +Use custom IDs for easier tracking: + +```java +var session = client.createSession(new SessionConfig() + .setSessionId("user-123-chat") + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + +System.out.println(session.getSessionId()); // "user-123-chat" +``` + +## Listing sessions + +```java +var sessions = client.listSessions().get(); +System.out.println(sessions); +// [SessionInfo{sessionId="user-123-chat", ...}, ...] +``` + +## Deleting sessions + +```java +// Delete a specific session +client.deleteSession("user-123-chat").get(); +``` + +## Managing session lifecycle with CompletableFuture + +Create and message sessions in parallel using `CompletableFuture.allOf`: + +```java +import java.util.concurrent.CompletableFuture; + +// Create all sessions in parallel +var f1 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)); +var f2 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)); +var f3 = client.createSession(new SessionConfig() + .setModel("claude-sonnet-4.5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)); + +CompletableFuture.allOf(f1, f2, f3).get(); + +var s1 = f1.get(); +var s2 = f2.get(); +var s3 = f3.get(); + +// Send messages in parallel +CompletableFuture.allOf( + s1.sendAndWait(new MessageOptions().setPrompt("Explain Java records")), + s2.sendAndWait(new MessageOptions().setPrompt("Explain sealed classes")), + s3.sendAndWait(new MessageOptions().setPrompt("Explain pattern matching")) +).get(); +``` + +## Providing a custom Executor + +Supply your own thread pool for parallel session work: + +```java +import java.util.concurrent.Executors; + +var executor = Executors.newFixedThreadPool(4); + +var client = new CopilotClient(new CopilotClientOptions() + .setExecutor(executor)); +client.start().get(); + +// Sessions now run on the custom executor +var session = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + +session.sendAndWait(new MessageOptions().setPrompt("Hello!")).get(); + +session.destroy().get(); +client.stop().get(); +executor.shutdown(); +``` + +## Use cases + +- **Multi-user applications**: One session per user +- **Multi-task workflows**: Separate sessions for different tasks +- **A/B testing**: Compare responses from different models diff --git a/cookbook/copilot-sdk/java/persisting-sessions.md b/cookbook/copilot-sdk/java/persisting-sessions.md new file mode 100644 index 00000000..5de9c0e2 --- /dev/null +++ b/cookbook/copilot-sdk/java/persisting-sessions.md @@ -0,0 +1,320 @@ +# Session Persistence and Resumption + +Save and restore conversation sessions across application restarts. + +> **Runnable example:** [recipe/PersistingSessions.java](recipe/PersistingSessions.java) +> +> ```bash +> jbang recipe/PersistingSessions.java +> ``` + +## Example scenario + +You want users to be able to continue a conversation even after closing and reopening your application. The Copilot SDK persists session state to disk automatically β€” you just need to provide a stable session ID and resume later. + +## Creating a session with a custom ID + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; + +public class CreateSessionWithId { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Create session with a memorable ID + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setSessionId("user-123-conversation") + .setModel("gpt-5") + ).get(); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println(msg.getData().content()) + ); + + session.sendAndWait(new MessageOptions() + .setPrompt("Let's discuss TypeScript generics")).get(); + + // Session ID is preserved + System.out.println("Session ID: " + session.getSessionId()); + + // Close session but keep data on disk + session.close(); + } + } +} +``` + +## Resuming a session + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.ResumeSessionConfig; + +public class ResumeSession { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Resume the previous session + var session = client.resumeSession( + "user-123-conversation", + new ResumeSessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + ).get(); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println(msg.getData().content()) + ); + + // Previous context is restored + session.sendAndWait(new MessageOptions() + .setPrompt("What were we discussing?")).get(); + + session.close(); + } + } +} +``` + +## Listing available sessions + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; + +public class ListSessions { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + var sessions = client.listSessions().get(); + for (var sessionInfo : sessions) { + System.out.println("Session: " + sessionInfo.getSessionId()); + } + } + } +} +``` + +## Deleting a session permanently + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; + +public class DeleteSession { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Remove session and all its data from disk + client.deleteSession("user-123-conversation").get(); + System.out.println("Session deleted"); + } + } +} +``` + +## Getting session history + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.events.UserMessageEvent; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.ResumeSessionConfig; + +public class SessionHistory { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + var session = client.resumeSession( + "user-123-conversation", + new ResumeSessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + ).get(); + + var messages = session.getMessages().get(); + for (var event : messages) { + if (event instanceof AssistantMessageEvent msg) { + System.out.printf("[assistant] %s%n", msg.getData().content()); + } else if (event instanceof UserMessageEvent userMsg) { + System.out.printf("[user] %s%n", userMsg.getData().content()); + } else { + System.out.printf("[%s]%n", event.getType()); + } + } + + session.close(); + } + } +} +``` + +## Complete example with session management + +This interactive example lets you create, resume, or list sessions from the command line. + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.json.*; +import java.util.Scanner; + +public class SessionManager { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient(); + var scanner = new Scanner(System.in)) { + + client.start().get(); + + System.out.println("Session Manager"); + System.out.println("1. Create new session"); + System.out.println("2. Resume existing session"); + System.out.println("3. List sessions"); + System.out.print("Choose an option: "); + + int choice = scanner.nextInt(); + scanner.nextLine(); + + switch (choice) { + case 1 -> { + System.out.print("Enter session ID: "); + String sessionId = scanner.nextLine(); + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setSessionId(sessionId) + .setModel("gpt-5") + ).get(); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nCopilot: " + msg.getData().content()) + ); + + System.out.println("Created session: " + sessionId); + chatLoop(session, scanner); + session.close(); + } + + case 2 -> { + System.out.print("Enter session ID to resume: "); + String resumeId = scanner.nextLine(); + try { + var session = client.resumeSession( + resumeId, + new ResumeSessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + ).get(); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nCopilot: " + msg.getData().content()) + ); + + System.out.println("Resumed session: " + resumeId); + chatLoop(session, scanner); + session.close(); + } catch (Exception ex) { + System.err.println("Failed to resume session: " + ex.getMessage()); + } + } + + case 3 -> { + var sessions = client.listSessions().get(); + System.out.println("\nAvailable sessions:"); + for (var s : sessions) { + System.out.println(" - " + s.getSessionId()); + } + } + + default -> System.out.println("Invalid choice"); + } + } + } + + static void chatLoop(Object session, Scanner scanner) throws Exception { + System.out.println("\nStart chatting (type 'exit' to quit):"); + while (true) { + System.out.print("\nYou: "); + String input = scanner.nextLine(); + if (input.equalsIgnoreCase("exit")) break; + + // Use reflection-free approach: cast to the session type + var s = (com.github.copilot.sdk.CopilotSession) session; + s.sendAndWait(new MessageOptions().setPrompt(input)).get(); + } + } +} +``` + +## Checking if a session exists + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.json.*; + +public class CheckSession { + public static boolean sessionExists(CopilotClient client, String sessionId) { + try { + var sessions = client.listSessions().get(); + return sessions.stream() + .anyMatch(s -> s.getSessionId().equals(sessionId)); + } catch (Exception ex) { + return false; + } + } + + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + String sessionId = "user-123-conversation"; + + if (sessionExists(client, sessionId)) { + System.out.println("Session exists, resuming..."); + var session = client.resumeSession( + sessionId, + new ResumeSessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + ).get(); + // ... use session ... + session.close(); + } else { + System.out.println("Session doesn't exist, creating new one..."); + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setSessionId(sessionId) + .setModel("gpt-5") + ).get(); + // ... use session ... + session.close(); + } + } + } +} +``` + +## Best practices + +1. **Use meaningful session IDs**: Include user ID or context in the session ID (e.g., `"user-123-chat"`, `"task-456-review"`) +2. **Handle missing sessions**: Check if a session exists before resuming β€” use `listSessions()` or catch the exception from `resumeSession()` +3. **Clean up old sessions**: Periodically delete sessions that are no longer needed with `deleteSession()` +4. **Error handling**: Always wrap resume operations in try-catch blocks β€” sessions may have been deleted or expired +5. **Workspace awareness**: Sessions are tied to workspace paths; ensure consistency when resuming across environments diff --git a/cookbook/copilot-sdk/java/pr-visualization.md b/cookbook/copilot-sdk/java/pr-visualization.md new file mode 100644 index 00000000..564a0c43 --- /dev/null +++ b/cookbook/copilot-sdk/java/pr-visualization.md @@ -0,0 +1,235 @@ +# Generating PR Age Charts + +Build an interactive CLI tool that visualizes pull request age distribution for a GitHub repository using Copilot's built-in capabilities. + +> **Runnable example:** [recipe/PRVisualization.java](recipe/PRVisualization.java) +> +> ```bash +> jbang recipe/PRVisualization.java +> ``` + +## Example scenario + +You want to understand how long PRs have been open in a repository. This tool detects the current Git repo or accepts a repo as input, then lets Copilot fetch PR data via the GitHub MCP Server and generate a chart image. + +## Usage + +```bash +# Auto-detect from current git repo +jbang recipe/PRVisualization.java + +# Specify a repo explicitly +jbang recipe/PRVisualization.java github/copilot-sdk +``` + +## Full example: PRVisualization.java + +```java +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.events.ToolExecutionStartEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; +import com.github.copilot.sdk.json.SystemMessageConfig; +import java.io.BufferedReader; +import java.io.EOFException; +import java.io.IOException; +import java.io.InputStreamReader; +import java.util.regex.Pattern; + +public class PRVisualization { + + public static void main(String[] args) throws Exception { + System.out.println("πŸ” PR Age Chart Generator\n"); + + // Determine the repository + String repo; + if (args.length > 0) { + repo = args[0]; + System.out.println("πŸ“¦ Using specified repo: " + repo); + } else if (isGitRepo()) { + String detected = getGitHubRemote(); + if (detected != null && !detected.isEmpty()) { + repo = detected; + System.out.println("πŸ“¦ Detected GitHub repo: " + repo); + } else { + System.out.println("⚠️ Git repo found but no GitHub remote detected."); + repo = promptForRepo(); + } + } else { + System.out.println("πŸ“ Not in a git repository."); + repo = promptForRepo(); + } + + if (repo == null || !repo.contains("/")) { + System.err.println("❌ Invalid repo format. Expected: owner/repo"); + System.exit(1); + } + + String[] parts = repo.split("/", 2); + String owner = parts[0]; + String repoName = parts[1]; + + // Create Copilot client + try (var client = new CopilotClient()) { + client.start().get(); + + String cwd = System.getProperty("user.dir"); + var systemMessage = String.format(""" + + You are analyzing pull requests for the GitHub repository: %s/%s + The current working directory is: %s + + + + - Use the GitHub MCP Server tools to fetch PR data + - Use your file and code execution tools to generate charts + - Save any generated images to the current working directory + - Be concise in your responses + + """, owner, repoName, cwd); + + var session = client.createSession( + new SessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5") + .setSystemMessage(new SystemMessageConfig().setContent(systemMessage)) + ).get(); + + // Set up event handling + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nπŸ€– " + msg.getData().content() + "\n") + ); + + session.on(ToolExecutionStartEvent.class, evt -> + System.out.println(" βš™οΈ " + evt.getData().toolName()) + ); + + // Initial prompt - let Copilot figure out the details + System.out.println("\nπŸ“Š Starting analysis...\n"); + + String prompt = String.format(""" + Fetch the open pull requests for %s/%s from the last week. + Calculate the age of each PR in days. + Then generate a bar chart image showing the distribution of PR ages + (group them into sensible buckets like <1 day, 1-3 days, etc.). + Save the chart as "pr-age-chart.png" in the current directory. + Finally, summarize the PR health - average age, oldest PR, and how many might be considered stale. + """, owner, repoName); + + session.send(new MessageOptions().setPrompt(prompt)); + + // Wait a bit for initial processing + Thread.sleep(10000); + + // Interactive loop + System.out.println("\nπŸ’‘ Ask follow-up questions or type \"exit\" to quit.\n"); + System.out.println("Examples:"); + System.out.println(" - \"Expand to the last month\""); + System.out.println(" - \"Show me the 5 oldest PRs\""); + System.out.println(" - \"Generate a pie chart instead\""); + System.out.println(" - \"Group by author instead of age\""); + System.out.println(); + + try (var reader = new BufferedReader(new InputStreamReader(System.in))) { + while (true) { + System.out.print("You: "); + String input = reader.readLine(); + if (input == null) break; + input = input.trim(); + + if (input.isEmpty()) continue; + if (input.equalsIgnoreCase("exit") || input.equalsIgnoreCase("quit")) { + System.out.println("πŸ‘‹ Goodbye!"); + break; + } + + session.send(new MessageOptions().setPrompt(input)); + Thread.sleep(2000); // Give time for response + } + } + + session.close(); + } + } + + // ============================================================================ + // Git & GitHub Detection + // ============================================================================ + + private static boolean isGitRepo() { + try { + Process proc = Runtime.getRuntime().exec(new String[]{"git", "rev-parse", "--git-dir"}); + return proc.waitFor() == 0; + } catch (Exception e) { + return false; + } + } + + private static String getGitHubRemote() { + try { + Process proc = Runtime.getRuntime().exec(new String[]{"git", "remote", "get-url", "origin"}); + try (BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()))) { + String remoteURL = reader.readLine(); + if (remoteURL == null) return null; + remoteURL = remoteURL.trim(); + + // Handle SSH: git@github.com:owner/repo.git + var sshPattern = Pattern.compile("git@github\\.com:(.+/.+?)(?:\\.git)?$"); + var sshMatcher = sshPattern.matcher(remoteURL); + if (sshMatcher.find()) { + return sshMatcher.group(1); + } + + // Handle HTTPS: https://github.com/owner/repo.git + var httpsPattern = Pattern.compile("https://github\\.com/(.+/.+?)(?:\\.git)?$"); + var httpsMatcher = httpsPattern.matcher(remoteURL); + if (httpsMatcher.find()) { + return httpsMatcher.group(1); + } + } + } catch (Exception e) { + // Ignore + } + return null; + } + + private static String promptForRepo() throws IOException { + BufferedReader reader = new BufferedReader(new InputStreamReader(System.in)); + System.out.print("Enter GitHub repo (owner/repo): "); + String line = reader.readLine(); + if (line == null) { + throw new EOFException("End of input while reading repository name"); + } + return line.trim(); + } +} +``` + +## How it works + +1. **Repository detection**: Checks command-line argument β†’ git remote β†’ prompts user +2. **No custom tools**: Relies entirely on Copilot CLI's built-in capabilities: + - **GitHub MCP Server** β€” Fetches PR data from GitHub + - **File tools** β€” Saves generated chart images + - **Code execution** β€” Generates charts using Python/matplotlib or other methods +3. **Interactive session**: After initial analysis, user can ask for adjustments + +## Why this approach? + +| Aspect | Custom Tools | Built-in Copilot | +| --------------- | ----------------- | --------------------------------- | +| Code complexity | High | **Minimal** | +| Maintenance | You maintain | **Copilot maintains** | +| Flexibility | Fixed logic | **AI decides best approach** | +| Chart types | What you coded | **Any type Copilot can generate** | +| Data grouping | Hardcoded buckets | **Intelligent grouping** | + +## Best practices + +1. **Start with auto-detection**: Let the tool detect the repository from the git remote before prompting the user +2. **Use system messages**: Provide context about the repo and working directory so Copilot can act autonomously +3. **Approve tool execution**: Use `PermissionHandler.APPROVE_ALL` to allow Copilot to run tools like the GitHub MCP Server without manual approval +4. **Interactive follow-ups**: Let users refine the analysis conversationally instead of requiring restarts +5. **Save artifacts locally**: Direct Copilot to save generated charts to the current directory for easy access diff --git a/cookbook/copilot-sdk/java/ralph-loop.md b/cookbook/copilot-sdk/java/ralph-loop.md new file mode 100644 index 00000000..84d77478 --- /dev/null +++ b/cookbook/copilot-sdk/java/ralph-loop.md @@ -0,0 +1,247 @@ +# Ralph Loop: Autonomous AI Task Loops + +Build autonomous coding loops where an AI agent picks tasks, implements them, validates against backpressure (tests, builds), commits, and repeats β€” each iteration in a fresh context window. + +> **Runnable example:** [recipe/RalphLoop.java](recipe/RalphLoop.java) +> +> ```bash +> jbang recipe/RalphLoop.java +> ``` + +## What is a Ralph Loop? + +A [Ralph loop](https://ghuntley.com/ralph/) is an autonomous development workflow where an AI agent iterates through tasks in isolated context windows. The key insight: **state lives on disk, not in the model's context**. Each iteration starts fresh, reads the current state from files, does one task, writes results back to disk, and exits. + +``` +β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” +β”‚ loop.sh β”‚ +β”‚ while true: β”‚ +β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ +β”‚ β”‚ Fresh session (isolated context) β”‚ β”‚ +β”‚ β”‚ β”‚ β”‚ +β”‚ β”‚ 1. Read PROMPT.md + AGENTS.md β”‚ β”‚ +β”‚ β”‚ 2. Study specs/* and code β”‚ β”‚ +β”‚ β”‚ 3. Pick next task from plan β”‚ β”‚ +β”‚ β”‚ 4. Implement + run tests β”‚ β”‚ +β”‚ β”‚ 5. Update plan, commit, exit β”‚ β”‚ +β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ +β”‚ ↻ next iteration (fresh context) β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +**Core principles:** + +- **Fresh context per iteration**: Each loop creates a new session β€” no context accumulation, always in the "smart zone" +- **Disk as shared state**: `IMPLEMENTATION_PLAN.md` persists between iterations and acts as the coordination mechanism +- **Backpressure steers quality**: Tests, builds, and lints reject bad work β€” the agent must fix issues before committing +- **Two modes**: PLANNING (gap analysis β†’ generate plan) and BUILDING (implement from plan) + +## Simple Version + +The minimal Ralph loop β€” the SDK equivalent of `while :; do cat PROMPT.md | copilot ; done`: + +```java +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.nio.file.*; + +public class SimpleRalphLoop { + public static void main(String[] args) throws Exception { + String promptFile = args.length > 0 ? args[0] : "PROMPT.md"; + int maxIterations = args.length > 1 ? Integer.parseInt(args[1]) : 50; + + try (var client = new CopilotClient()) { + client.start().get(); + + String prompt = Files.readString(Path.of(promptFile)); + + for (int i = 1; i <= maxIterations; i++) { + System.out.printf("%n=== Iteration %d/%d ===%n", i, maxIterations); + + // Fresh session each iteration β€” context isolation is the point + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5.1-codex-mini") + .setWorkingDirectory(System.getProperty("user.dir")) + ).get(); + + try { + session.sendAndWait(new MessageOptions().setPrompt(prompt)).get(); + } finally { + session.close(); + } + + System.out.printf("Iteration %d complete.%n", i); + } + } + } +} +``` + +This is all you need to get started. The prompt file tells the agent what to do; the agent reads project files, does work, commits, and exits. The loop restarts with a clean slate. + +## Ideal Version + +The full Ralph pattern with planning and building modes, matching the [Ralph Playbook](https://github.com/ClaytonFarr/ralph-playbook) architecture: + +```java +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.nio.file.*; +import java.util.Arrays; + +public class RalphLoop { + public static void main(String[] args) throws Exception { + // Parse CLI args: jbang RalphLoop.java [plan] [max_iterations] + boolean planMode = Arrays.asList(args).contains("plan"); + String mode = planMode ? "plan" : "build"; + int maxIterations = Arrays.stream(args) + .filter(a -> a.matches("\\d+")) + .findFirst() + .map(Integer::parseInt) + .orElse(50); + + String promptFile = planMode ? "PROMPT_plan.md" : "PROMPT_build.md"; + System.out.printf("Mode: %s | Prompt: %s%n", mode, promptFile); + + try (var client = new CopilotClient()) { + client.start().get(); + + String prompt = Files.readString(Path.of(promptFile)); + + for (int i = 1; i <= maxIterations; i++) { + System.out.printf("%n=== Iteration %d/%d ===%n", i, maxIterations); + + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5.1-codex-mini") + .setWorkingDirectory(System.getProperty("user.dir")) + ).get(); + + // Log tool usage for visibility + session.on(ToolExecutionStartEvent.class, + ev -> System.out.printf(" βš™ %s%n", ev.getData().toolName())); + + try { + session.sendAndWait(new MessageOptions().setPrompt(prompt)).get(); + } finally { + session.close(); + } + + System.out.printf("Iteration %d complete.%n", i); + } + } + } +} +``` + +### Required Project Files + +The ideal version expects this file structure in your project: + +``` +project-root/ +β”œβ”€β”€ PROMPT_plan.md # Planning mode instructions +β”œβ”€β”€ PROMPT_build.md # Building mode instructions +β”œβ”€β”€ AGENTS.md # Operational guide (build/test commands) +β”œβ”€β”€ IMPLEMENTATION_PLAN.md # Task list (generated by planning mode) +β”œβ”€β”€ specs/ # Requirement specs (one per topic) +β”‚ β”œβ”€β”€ auth.md +β”‚ └── data-pipeline.md +└── src/ # Your source code +``` + +### Example `PROMPT_plan.md` + +```markdown +0a. Study `specs/*` to learn the application specifications. +0b. Study IMPLEMENTATION_PLAN.md (if present) to understand the plan so far. +0c. Study `src/` to understand existing code and shared utilities. + +1. Compare specs against code (gap analysis). Create or update + IMPLEMENTATION_PLAN.md as a prioritized bullet-point list of tasks + yet to be implemented. Do NOT implement anything. + +IMPORTANT: Do NOT assume functionality is missing β€” search the +codebase first to confirm. Prefer updating existing utilities over +creating ad-hoc copies. +``` + +### Example `PROMPT_build.md` + +```markdown +0a. Study `specs/*` to learn the application specifications. +0b. Study IMPLEMENTATION_PLAN.md. +0c. Study `src/` for reference. + +1. Choose the most important item from IMPLEMENTATION_PLAN.md. Before + making changes, search the codebase (don't assume not implemented). +2. After implementing, run the tests. If functionality is missing, add it. +3. When you discover issues, update IMPLEMENTATION_PLAN.md immediately. +4. When tests pass, update IMPLEMENTATION_PLAN.md, then `git add -A` + then `git commit` with a descriptive message. + +5. When authoring documentation, capture the why. +6. Implement completely. No placeholders or stubs. +7. Keep IMPLEMENTATION_PLAN.md current β€” future iterations depend on it. +``` + +### Example `AGENTS.md` + +Keep this brief (~60 lines). It's loaded every iteration, so bloat wastes context. + +```markdown +## Build & Run + +mvn compile + +## Validation + +- Tests: `mvn test` +- Typecheck: `mvn compile` +- Lint: `mvn checkstyle:check` +``` + +## Best Practices + +1. **Fresh context per iteration**: Never accumulate context across iterations β€” that's the whole point +2. **Disk is your database**: `IMPLEMENTATION_PLAN.md` is shared state between isolated sessions +3. **Backpressure is essential**: Tests, builds, lints in `AGENTS.md` β€” the agent must pass them before committing +4. **Start with PLANNING mode**: Generate the plan first, then switch to BUILDING +5. **Observe and tune**: Watch early iterations, add guardrails to prompts when the agent fails in specific ways +6. **The plan is disposable**: If the agent goes off track, delete `IMPLEMENTATION_PLAN.md` and re-plan +7. **Keep `AGENTS.md` brief**: It's loaded every iteration β€” operational info only, no progress notes +8. **Use a sandbox**: The agent runs autonomously with full tool access β€” isolate it +9. **Set `workingDirectory`**: Pin the session to your project root so tool operations resolve paths correctly +10. **Auto-approve permissions**: Use `PermissionHandler.APPROVE_ALL` to allow tool calls without interrupting the loop + +## When to Use a Ralph Loop + +**Good for:** + +- Implementing features from specs with test-driven validation +- Large refactors broken into many small tasks +- Unattended, long-running development with clear requirements +- Any work where backpressure (tests/builds) can verify correctness + +**Not good for:** + +- Tasks requiring human judgment mid-loop +- One-shot operations that don't benefit from iteration +- Vague requirements without testable acceptance criteria +- Exploratory prototyping where direction isn't clear + +## See Also + +- [Error Handling](error-handling.md) β€” timeout patterns and graceful shutdown for long-running sessions +- [Persisting Sessions](persisting-sessions.md) β€” save and resume sessions across restarts diff --git a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java new file mode 100644 index 00000000..917d684b --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java @@ -0,0 +1,123 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.io.*; +import java.util.*; +import java.util.concurrent.*; + +/** + * Accessibility Report Generator β€” analyzes web pages using the Playwright MCP server + * and generates WCAG-compliant accessibility reports. + * + * Usage: + * jbang AccessibilityReport.java + */ +public class AccessibilityReport { + public static void main(String[] args) throws Exception { + System.out.println("=== Accessibility Report Generator ===\n"); + + var reader = new BufferedReader(new InputStreamReader(System.in)); + + System.out.print("Enter URL to analyze: "); + String url = reader.readLine().trim(); + if (url.isEmpty()) { + System.out.println("No URL provided. Exiting."); + return; + } + if (!url.startsWith("http://") && !url.startsWith("https://")) { + url = "https://" + url; + } + + System.out.printf("%nAnalyzing: %s%n", url); + System.out.println("Please wait...\n"); + + try (var client = new CopilotClient()) { + client.start().get(); + + // Configure Playwright MCP server for browser automation + var mcpConfig = new McpServerConfig() + .setType("local") + .setCommand("npx") + .setArgs(List.of("@playwright/mcp@latest")) + .setTools(List.of("*")); + + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("claude-opus-4.6") + .setStreaming(true) + .setMcpServers(Map.of("playwright", mcpConfig)) + ).get(); + + // Stream output token-by-token + var idleLatch = new CountDownLatch(1); + + session.on(AssistantMessageDeltaEvent.class, + ev -> System.out.print(ev.getData().deltaContent())); + + session.on(SessionIdleEvent.class, + ev -> idleLatch.countDown()); + + session.on(SessionErrorEvent.class, ev -> { + System.err.printf("%nError: %s%n", ev.getData().message()); + idleLatch.countDown(); + }); + + String prompt = """ + Use the Playwright MCP server to analyze the accessibility of this webpage: %s + + Please: + 1. Navigate to the URL using playwright-browser_navigate + 2. Take an accessibility snapshot using playwright-browser_snapshot + 3. Analyze the snapshot and provide a detailed accessibility report + + Format the report with emoji indicators: + - πŸ“Š Accessibility Report header + - βœ… What's Working Well (table with Category, Status, Details) + - ⚠️ Issues Found (table with Severity, Issue, WCAG Criterion, Recommendation) + - πŸ“‹ Stats Summary (links, headings, focusable elements, landmarks) + - βš™οΈ Priority Recommendations + + Use βœ… for pass, πŸ”΄ for high severity issues, 🟑 for medium severity, ❌ for missing items. + Include actual findings from the page analysis. + """.formatted(url); + + session.send(new MessageOptions().setPrompt(prompt)); + idleLatch.await(); + + System.out.println("\n\n=== Report Complete ===\n"); + + // Prompt user for test generation + System.out.print("Would you like to generate Playwright accessibility tests? (y/n): "); + String generateTests = reader.readLine().trim(); + + if (generateTests.equalsIgnoreCase("y") || generateTests.equalsIgnoreCase("yes")) { + var testLatch = new CountDownLatch(1); + + session.on(SessionIdleEvent.class, + ev -> testLatch.countDown()); + + String testPrompt = """ + Based on the accessibility report you just generated for %s, + create Playwright accessibility tests in Java. + + Include tests for: lang attribute, title, heading hierarchy, alt text, + landmarks, skip navigation, focus indicators, and touch targets. + Use Playwright's accessibility testing features with helpful comments. + Output the complete test file. + """.formatted(url); + + System.out.println("\nGenerating accessibility tests...\n"); + session.send(new MessageOptions().setPrompt(testPrompt)); + testLatch.await(); + + System.out.println("\n\n=== Tests Generated ==="); + } + + session.close(); + } + } +} diff --git a/cookbook/copilot-sdk/java/recipe/ErrorHandling.java b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java new file mode 100644 index 00000000..4765adc4 --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java @@ -0,0 +1,34 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.util.concurrent.ExecutionException; + +public class ErrorHandling { + public static void main(String[] args) { + try (var client = new CopilotClient()) { + client.start().get(); + + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5")).get(); + + session.on(AssistantMessageEvent.class, + msg -> System.out.println(msg.getData().content())); + + session.sendAndWait( + new MessageOptions().setPrompt("Hello!")).get(); + + session.close(); + } catch (ExecutionException ex) { + System.err.println("Error: " + ex.getCause().getMessage()); + ex.getCause().printStackTrace(); + } catch (Exception ex) { + System.err.println("Error: " + ex.getMessage()); + ex.printStackTrace(); + } + } +} diff --git a/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java b/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java new file mode 100644 index 00000000..0b022eee --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java @@ -0,0 +1,64 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.events.SessionIdleEvent; +import com.github.copilot.sdk.events.ToolExecutionCompleteEvent; +import com.github.copilot.sdk.events.ToolExecutionStartEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; +import java.nio.file.Paths; +import java.util.concurrent.CountDownLatch; + +public class ManagingLocalFiles { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Create session + var session = client.createSession( + new SessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL).setModel("gpt-5")).get(); + + // Set up event handlers + var done = new CountDownLatch(1); + + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nCopilot: " + msg.getData().content()) + ); + + session.on(ToolExecutionStartEvent.class, evt -> + System.out.println(" β†’ Running: " + evt.getData().toolName()) + ); + + session.on(ToolExecutionCompleteEvent.class, evt -> + System.out.println(" βœ“ Completed: " + evt.getData().toolCallId()) + ); + + session.on(SessionIdleEvent.class, evt -> done.countDown()); + + // Ask Copilot to organize files - using a safe example folder + // For real use, replace with your target folder + String targetFolder = args.length > 0 ? args[0] : + System.getProperty("java.io.tmpdir") + "/example-files"; + + String prompt = String.format(""" + Analyze the files in "%s" and show how you would organize them into subfolders. + + 1. First, list all files and their metadata + 2. Preview grouping by file extension + 3. Suggest appropriate subfolders (e.g., "images", "documents", "videos") + + IMPORTANT: DO NOT move any files. Only show the plan. + """, targetFolder); + + session.send(new MessageOptions().setPrompt(prompt)); + + // Wait for completion + done.await(); + + session.close(); + } + } +} diff --git a/cookbook/copilot-sdk/java/recipe/MultipleSessions.java b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java new file mode 100644 index 00000000..bd3b27f1 --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java @@ -0,0 +1,37 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.json.*; +import java.util.concurrent.CompletableFuture; + +public class MultipleSessions { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + var config = new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL); + + // Create 3 sessions in parallel + var f1 = client.createSession(config); + var f2 = client.createSession(config); + var f3 = client.createSession(new SessionConfig() + .setModel("claude-sonnet-4.5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)); + CompletableFuture.allOf(f1, f2, f3).get(); + + var s1 = f1.get(); var s2 = f2.get(); var s3 = f3.get(); + + // Send a message to each session + System.out.println("S1: " + s1.sendAndWait(new MessageOptions().setPrompt("Explain Java records")).get().getMessage()); + System.out.println("S2: " + s2.sendAndWait(new MessageOptions().setPrompt("Explain sealed classes")).get().getMessage()); + System.out.println("S3: " + s3.sendAndWait(new MessageOptions().setPrompt("Explain pattern matching")).get().getMessage()); + + // Clean up + s1.destroy().get(); s2.destroy().get(); s3.destroy().get(); + client.stop().get(); + } + } +} diff --git a/cookbook/copilot-sdk/java/recipe/PRVisualization.java b/cookbook/copilot-sdk/java/recipe/PRVisualization.java new file mode 100644 index 00000000..4d46d0dc --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/PRVisualization.java @@ -0,0 +1,182 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.CopilotClient; +import com.github.copilot.sdk.events.AssistantMessageEvent; +import com.github.copilot.sdk.events.ToolExecutionStartEvent; +import com.github.copilot.sdk.json.MessageOptions; +import com.github.copilot.sdk.json.PermissionHandler; +import com.github.copilot.sdk.json.SessionConfig; +import com.github.copilot.sdk.json.SystemMessageConfig; +import java.io.BufferedReader; +import java.io.EOFException; +import java.io.IOException; +import java.io.InputStreamReader; +import java.util.regex.Pattern; + +public class PRVisualization { + + public static void main(String[] args) throws Exception { + System.out.println("πŸ” PR Age Chart Generator\n"); + + // Determine the repository + String repo; + if (args.length > 0) { + repo = args[0]; + System.out.println("πŸ“¦ Using specified repo: " + repo); + } else if (isGitRepo()) { + String detected = getGitHubRemote(); + if (detected != null && !detected.isEmpty()) { + repo = detected; + System.out.println("πŸ“¦ Detected GitHub repo: " + repo); + } else { + System.out.println("⚠️ Git repo found but no GitHub remote detected."); + repo = promptForRepo(); + } + } else { + System.out.println("πŸ“ Not in a git repository."); + repo = promptForRepo(); + } + + if (repo == null || !repo.contains("/")) { + System.err.println("❌ Invalid repo format. Expected: owner/repo"); + System.exit(1); + } + + String[] parts = repo.split("/", 2); + String owner = parts[0]; + String repoName = parts[1]; + + // Create Copilot client + try (var client = new CopilotClient()) { + client.start().get(); + + String cwd = System.getProperty("user.dir"); + var systemMessage = String.format(""" + + You are analyzing pull requests for the GitHub repository: %s/%s + The current working directory is: %s + + + + - Use the GitHub MCP Server tools to fetch PR data + - Use your file and code execution tools to generate charts + - Save any generated images to the current working directory + - Be concise in your responses + + """, owner, repoName, cwd); + + var session = client.createSession( + new SessionConfig().setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5") + .setSystemMessage(new SystemMessageConfig().setContent(systemMessage)) + ).get(); + + // Set up event handling + session.on(AssistantMessageEvent.class, msg -> + System.out.println("\nπŸ€– " + msg.getData().content() + "\n") + ); + + session.on(ToolExecutionStartEvent.class, evt -> + System.out.println(" βš™οΈ " + evt.getData().toolName()) + ); + + // Initial prompt - let Copilot figure out the details + System.out.println("\nπŸ“Š Starting analysis...\n"); + + String prompt = String.format(""" + Fetch the open pull requests for %s/%s from the last week. + Calculate the age of each PR in days. + Then generate a bar chart image showing the distribution of PR ages + (group them into sensible buckets like <1 day, 1-3 days, etc.). + Save the chart as "pr-age-chart.png" in the current directory. + Finally, summarize the PR health - average age, oldest PR, and how many might be considered stale. + """, owner, repoName); + + session.send(new MessageOptions().setPrompt(prompt)); + + // Wait a bit for initial processing + Thread.sleep(10000); + + // Interactive loop + System.out.println("\nπŸ’‘ Ask follow-up questions or type \"exit\" to quit.\n"); + System.out.println("Examples:"); + System.out.println(" - \"Expand to the last month\""); + System.out.println(" - \"Show me the 5 oldest PRs\""); + System.out.println(" - \"Generate a pie chart instead\""); + System.out.println(" - \"Group by author instead of age\""); + System.out.println(); + + try (var reader = new BufferedReader(new InputStreamReader(System.in))) { + while (true) { + System.out.print("You: "); + String input = reader.readLine(); + if (input == null) break; + input = input.trim(); + + if (input.isEmpty()) continue; + if (input.equalsIgnoreCase("exit") || input.equalsIgnoreCase("quit")) { + System.out.println("πŸ‘‹ Goodbye!"); + break; + } + + session.send(new MessageOptions().setPrompt(input)); + Thread.sleep(2000); // Give time for response + } + } + + session.close(); + } + } + + // ============================================================================ + // Git & GitHub Detection + // ============================================================================ + + private static boolean isGitRepo() { + try { + Process proc = Runtime.getRuntime().exec(new String[]{"git", "rev-parse", "--git-dir"}); + return proc.waitFor() == 0; + } catch (Exception e) { + return false; + } + } + + private static String getGitHubRemote() { + try { + Process proc = Runtime.getRuntime().exec(new String[]{"git", "remote", "get-url", "origin"}); + try (BufferedReader reader = new BufferedReader(new InputStreamReader(proc.getInputStream()))) { + String remoteURL = reader.readLine(); + if (remoteURL == null) return null; + remoteURL = remoteURL.trim(); + + // Handle SSH: git@github.com:owner/repo.git + var sshPattern = Pattern.compile("git@github\\.com:(.+/.+?)(?:\\.git)?$"); + var sshMatcher = sshPattern.matcher(remoteURL); + if (sshMatcher.find()) { + return sshMatcher.group(1); + } + + // Handle HTTPS: https://github.com/owner/repo.git + var httpsPattern = Pattern.compile("https://github\\.com/(.+/.+?)(?:\\.git)?$"); + var httpsMatcher = httpsPattern.matcher(remoteURL); + if (httpsMatcher.find()) { + return httpsMatcher.group(1); + } + } + } catch (Exception e) { + // Ignore + } + return null; + } + + private static String promptForRepo() throws IOException { + BufferedReader reader = new BufferedReader(new InputStreamReader(System.in)); + System.out.print("Enter GitHub repo (owner/repo): "); + String line = reader.readLine(); + if (line == null) { + throw new EOFException("End of input while reading repository name"); + } + return line.trim(); + } +} diff --git a/cookbook/copilot-sdk/java/recipe/PersistingSessions.java b/cookbook/copilot-sdk/java/recipe/PersistingSessions.java new file mode 100644 index 00000000..89663908 --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/PersistingSessions.java @@ -0,0 +1,34 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; + +public class PersistingSessions { + public static void main(String[] args) throws Exception { + try (var client = new CopilotClient()) { + client.start().get(); + + // Create a session with a custom ID so we can resume it later + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setSessionId("user-123-conversation") + .setModel("gpt-5") + ).get(); + + session.on(AssistantMessageEvent.class, + msg -> System.out.println(msg.getData().content())); + + session.sendAndWait(new MessageOptions() + .setPrompt("Let's discuss TypeScript generics")).get(); + + System.out.println("\nSession ID: " + session.getSessionId()); + System.out.println("Session closed β€” data persisted to disk."); + + // Close session but keep data on disk for later resumption + session.close(); + } + } +} diff --git a/cookbook/copilot-sdk/java/recipe/README.md b/cookbook/copilot-sdk/java/recipe/README.md new file mode 100644 index 00000000..232d156d --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/README.md @@ -0,0 +1,71 @@ +# Runnable Recipe Examples + +This folder contains standalone, executable Java examples for each cookbook recipe. Each file can be run directly with [JBang](https://www.jbang.dev/) β€” no project setup required. + +## Prerequisites + +- Java 17 or later +- JBang installed: + +```bash +# macOS (using Homebrew) +brew install jbangdev/tap/jbang + +# Linux/macOS (using curl) +curl -Ls https://sh.jbang.dev | bash -s - app setup + +# Windows (using Scoop) +scoop install jbang +``` + +For other installation methods, see the [JBang installation guide](https://www.jbang.dev/download/). + +## Running Examples + +Each `.java` file is a complete, runnable program. Simply use: + +```bash +jbang .java +``` + +### Available Recipes + +| Recipe | Command | Description | +| -------------------- | ------------------------------------ | ------------------------------------------ | +| Error Handling | `jbang ErrorHandling.java` | Demonstrates error handling patterns | +| Multiple Sessions | `jbang MultipleSessions.java` | Manages multiple independent conversations | +| Managing Local Files | `jbang ManagingLocalFiles.java` | Organizes files using AI grouping | +| PR Visualization | `jbang PRVisualization.java` | Generates PR age charts | +| Persisting Sessions | `jbang PersistingSessions.java` | Save and resume sessions across restarts | +| Ralph Loop | `jbang RalphLoop.java` | Autonomous AI task loop | +| Accessibility Report | `jbang AccessibilityReport.java` | WCAG accessibility report generator | + +### Examples with Arguments + +**PR Visualization with specific repo:** + +```bash +jbang PRVisualization.java github/copilot-sdk +``` + +**Managing Local Files with specific folder:** + +```bash +jbang ManagingLocalFiles.java /path/to/your/folder +``` + +**Ralph Loop in planning mode:** + +```bash +jbang RalphLoop.java plan 5 +``` + +## Why JBang? + +JBang lets you run Java files as scripts β€” no `pom.xml`, no `build.gradle`, no project scaffolding. Dependencies are declared inline with `//DEPS` comments and resolved automatically. + +## Learning Resources + +- [JBang Documentation](https://www.jbang.dev/documentation/guide/latest/) +- [GitHub Copilot SDK for Java](https://github.com/github/copilot-sdk-java) +- [Parent Cookbook](../README.md) diff --git a/cookbook/copilot-sdk/java/recipe/RalphLoop.java b/cookbook/copilot-sdk/java/recipe/RalphLoop.java new file mode 100644 index 00000000..99e04655 --- /dev/null +++ b/cookbook/copilot-sdk/java/recipe/RalphLoop.java @@ -0,0 +1,55 @@ +///usr/bin/env jbang "$0" "$@" ; exit $? +//DEPS com.github:copilot-sdk-java:0.2.1-java.1 + +import com.github.copilot.sdk.*; +import com.github.copilot.sdk.events.*; +import com.github.copilot.sdk.json.*; +import java.nio.file.*; + +/** + * Simple Ralph Loop β€” reads PROMPT.md and runs it in a fresh session each iteration. + * + * Usage: + * jbang RalphLoop.java # defaults: PROMPT.md, 50 iterations + * jbang RalphLoop.java PROMPT.md 20 # custom prompt file, 20 iterations + */ +public class RalphLoop { + public static void main(String[] args) throws Exception { + String promptFile = args.length > 0 ? args[0] : "PROMPT.md"; + int maxIterations = args.length > 1 ? Integer.parseInt(args[1]) : 50; + + System.out.printf("Ralph Loop β€” prompt: %s, max iterations: %d%n", promptFile, maxIterations); + + try (var client = new CopilotClient()) { + client.start().get(); + + String prompt = Files.readString(Path.of(promptFile)); + + for (int i = 1; i <= maxIterations; i++) { + System.out.printf("%n=== Iteration %d/%d ===%n", i, maxIterations); + + // Fresh session each iteration β€” context isolation is the point + var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5.1-codex-mini") + .setWorkingDirectory(System.getProperty("user.dir")) + ).get(); + + // Log tool usage for visibility + session.on(ToolExecutionStartEvent.class, + ev -> System.out.printf(" βš™ %s%n", ev.getData().toolName())); + + try { + session.sendAndWait(new MessageOptions().setPrompt(prompt)).get(); + } finally { + session.close(); + } + + System.out.printf("Iteration %d complete.%n", i); + } + } + + System.out.println("\nAll iterations complete."); + } +} From 57b27994085ae51ecf8fae4d082828899d77e7bf Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 14:09:17 -0400 Subject: [PATCH 02/16] Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- cookbook/copilot-sdk/README.md | 2 +- .../copilot-sdk/java/accessibility-report.md | 2 +- .../copilot-sdk/java/multiple-sessions.md | 56 ++++++++++--------- .../java/recipe/AccessibilityReport.java | 7 ++- .../java/recipe/ErrorHandling.java | 10 +++- .../java/recipe/MultipleSessions.java | 6 +- cookbook/copilot-sdk/java/recipe/README.md | 4 +- 7 files changed, 50 insertions(+), 37 deletions(-) diff --git a/cookbook/copilot-sdk/README.md b/cookbook/copilot-sdk/README.md index d9a127f9..c740200a 100644 --- a/cookbook/copilot-sdk/README.md +++ b/cookbook/copilot-sdk/README.md @@ -97,7 +97,7 @@ go run .go ### Java ```bash -cd java/cookbook/recipe +cd java/recipe jbang .java ``` diff --git a/cookbook/copilot-sdk/java/accessibility-report.md b/cookbook/copilot-sdk/java/accessibility-report.md index 2b6abbe6..785b3a42 100644 --- a/cookbook/copilot-sdk/java/accessibility-report.md +++ b/cookbook/copilot-sdk/java/accessibility-report.md @@ -27,7 +27,7 @@ npx --version ## Usage ```bash -jbang AccessibilityReport.java +jbang recipe/AccessibilityReport.java # Enter a URL when prompted ``` diff --git a/cookbook/copilot-sdk/java/multiple-sessions.md b/cookbook/copilot-sdk/java/multiple-sessions.md index 585ee5fd..3ac0d785 100644 --- a/cookbook/copilot-sdk/java/multiple-sessions.md +++ b/cookbook/copilot-sdk/java/multiple-sessions.md @@ -5,7 +5,7 @@ Manage multiple independent conversations simultaneously. > **Runnable example:** [recipe/MultipleSessions.java](recipe/MultipleSessions.java) > > ```bash -> jbang MultipleSessions.java +> jbang recipe/MultipleSessions.java > ``` ## Example scenario @@ -21,35 +21,39 @@ You need to run multiple conversations in parallel, each with its own context an import com.github.copilot.sdk.*; import com.github.copilot.sdk.json.*; -var client = new CopilotClient(); -client.start().get(); +public class MultipleSessions { + public static void main(String[] args) throws Exception { + var client = new CopilotClient(); + client.start().get(); -// Create multiple independent sessions -var session1 = client.createSession(new SessionConfig() - .setModel("gpt-5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); -var session2 = client.createSession(new SessionConfig() - .setModel("gpt-5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); -var session3 = client.createSession(new SessionConfig() - .setModel("claude-sonnet-4.5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + // Create multiple independent sessions + var session1 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + var session2 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + var session3 = client.createSession(new SessionConfig() + .setModel("claude-sonnet-4.5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); -// Each session maintains its own conversation history -session1.sendAndWait(new MessageOptions().setPrompt("You are helping with a Python project")).get(); -session2.sendAndWait(new MessageOptions().setPrompt("You are helping with a TypeScript project")).get(); -session3.sendAndWait(new MessageOptions().setPrompt("You are helping with a Go project")).get(); + // Each session maintains its own conversation history + session1.sendAndWait(new MessageOptions().setPrompt("You are helping with a Python project")).get(); + session2.sendAndWait(new MessageOptions().setPrompt("You are helping with a TypeScript project")).get(); + session3.sendAndWait(new MessageOptions().setPrompt("You are helping with a Go project")).get(); -// Follow-up messages stay in their respective contexts -session1.sendAndWait(new MessageOptions().setPrompt("How do I create a virtual environment?")).get(); -session2.sendAndWait(new MessageOptions().setPrompt("How do I set up tsconfig?")).get(); -session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); + // Follow-up messages stay in their respective contexts + session1.sendAndWait(new MessageOptions().setPrompt("How do I create a virtual environment?")).get(); + session2.sendAndWait(new MessageOptions().setPrompt("How do I set up tsconfig?")).get(); + session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); -// Clean up all sessions -session1.destroy().get(); -session2.destroy().get(); -session3.destroy().get(); -client.stop().get(); + // Clean up all sessions + session1.destroy().get(); + session2.destroy().get(); + session3.destroy().get(); + client.stop().get(); + } +} ``` ## Custom session IDs diff --git a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java index 917d684b..748e5e09 100644 --- a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java +++ b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java @@ -22,7 +22,12 @@ public class AccessibilityReport { var reader = new BufferedReader(new InputStreamReader(System.in)); System.out.print("Enter URL to analyze: "); - String url = reader.readLine().trim(); + String urlLine = reader.readLine(); + if (urlLine == null) { + System.out.println("No URL provided. Exiting."); + return; + } + String url = urlLine.trim(); if (url.isEmpty()) { System.out.println("No URL provided. Exiting."); return; diff --git a/cookbook/copilot-sdk/java/recipe/ErrorHandling.java b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java index 4765adc4..56813355 100644 --- a/cookbook/copilot-sdk/java/recipe/ErrorHandling.java +++ b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java @@ -24,8 +24,14 @@ public class ErrorHandling { session.close(); } catch (ExecutionException ex) { - System.err.println("Error: " + ex.getCause().getMessage()); - ex.getCause().printStackTrace(); + Throwable cause = ex.getCause(); + Throwable error = cause != null ? cause : ex; + System.err.println("Error: " + error.getMessage()); + error.printStackTrace(); + } catch (InterruptedException ex) { + Thread.currentThread().interrupt(); + System.err.println("Interrupted: " + ex.getMessage()); + ex.printStackTrace(); } catch (Exception ex) { System.err.println("Error: " + ex.getMessage()); ex.printStackTrace(); diff --git a/cookbook/copilot-sdk/java/recipe/MultipleSessions.java b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java index bd3b27f1..2512bdaf 100644 --- a/cookbook/copilot-sdk/java/recipe/MultipleSessions.java +++ b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java @@ -25,9 +25,9 @@ public class MultipleSessions { var s1 = f1.get(); var s2 = f2.get(); var s3 = f3.get(); // Send a message to each session - System.out.println("S1: " + s1.sendAndWait(new MessageOptions().setPrompt("Explain Java records")).get().getMessage()); - System.out.println("S2: " + s2.sendAndWait(new MessageOptions().setPrompt("Explain sealed classes")).get().getMessage()); - System.out.println("S3: " + s3.sendAndWait(new MessageOptions().setPrompt("Explain pattern matching")).get().getMessage()); + System.out.println("S1: " + s1.sendAndWait(new MessageOptions().setPrompt("Explain Java records")).get().getData().content()); + System.out.println("S2: " + s2.sendAndWait(new MessageOptions().setPrompt("Explain sealed classes")).get().getData().content()); + System.out.println("S3: " + s3.sendAndWait(new MessageOptions().setPrompt("Explain pattern matching")).get().getData().content()); // Clean up s1.destroy().get(); s2.destroy().get(); s3.destroy().get(); diff --git a/cookbook/copilot-sdk/java/recipe/README.md b/cookbook/copilot-sdk/java/recipe/README.md index 232d156d..5726a67b 100644 --- a/cookbook/copilot-sdk/java/recipe/README.md +++ b/cookbook/copilot-sdk/java/recipe/README.md @@ -54,10 +54,8 @@ jbang PRVisualization.java github/copilot-sdk jbang ManagingLocalFiles.java /path/to/your/folder ``` -**Ralph Loop in planning mode:** +**Ralph Loop with a prompt file:** -```bash -jbang RalphLoop.java plan 5 ``` ## Why JBang? From 0f06f346be6e8321a52b3373a8c78f0aa43db9e1 Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 14:12:08 -0400 Subject: [PATCH 03/16] Address review feedback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Replace Thread.sleep with sendAndWait in PRVisualization - Fix top-level statements in multiple-sessions.md (wrap in class) - Fix .getMessage() β†’ .getData().content() in MultipleSessions.java - Guard against null readLine() in AccessibilityReport.java - Add null-safe getCause() + InterruptedException handling in ErrorHandling.java - Fix paths: jbang commands now include recipe/ prefix - Fix root README path: cd java/recipe (not java/cookbook/recipe) - Fix recipe/README.md ralph-loop CLI example --- .../copilot-sdk/java/multiple-sessions.md | 50 +++++++++---------- cookbook/copilot-sdk/java/pr-visualization.md | 8 +-- .../java/recipe/PRVisualization.java | 8 +-- cookbook/copilot-sdk/java/recipe/README.md | 4 +- 4 files changed, 32 insertions(+), 38 deletions(-) diff --git a/cookbook/copilot-sdk/java/multiple-sessions.md b/cookbook/copilot-sdk/java/multiple-sessions.md index 3ac0d785..b44a4d76 100644 --- a/cookbook/copilot-sdk/java/multiple-sessions.md +++ b/cookbook/copilot-sdk/java/multiple-sessions.md @@ -23,35 +23,35 @@ import com.github.copilot.sdk.json.*; public class MultipleSessions { public static void main(String[] args) throws Exception { - var client = new CopilotClient(); - client.start().get(); + try (var client = new CopilotClient()) { + client.start().get(); - // Create multiple independent sessions - var session1 = client.createSession(new SessionConfig() - .setModel("gpt-5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); - var session2 = client.createSession(new SessionConfig() - .setModel("gpt-5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); - var session3 = client.createSession(new SessionConfig() - .setModel("claude-sonnet-4.5") - .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + // Create multiple independent sessions + var session1 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + var session2 = client.createSession(new SessionConfig() + .setModel("gpt-5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); + var session3 = client.createSession(new SessionConfig() + .setModel("claude-sonnet-4.5") + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL)).get(); - // Each session maintains its own conversation history - session1.sendAndWait(new MessageOptions().setPrompt("You are helping with a Python project")).get(); - session2.sendAndWait(new MessageOptions().setPrompt("You are helping with a TypeScript project")).get(); - session3.sendAndWait(new MessageOptions().setPrompt("You are helping with a Go project")).get(); + // Each session maintains its own conversation history + session1.sendAndWait(new MessageOptions().setPrompt("You are helping with a Python project")).get(); + session2.sendAndWait(new MessageOptions().setPrompt("You are helping with a TypeScript project")).get(); + session3.sendAndWait(new MessageOptions().setPrompt("You are helping with a Go project")).get(); - // Follow-up messages stay in their respective contexts - session1.sendAndWait(new MessageOptions().setPrompt("How do I create a virtual environment?")).get(); - session2.sendAndWait(new MessageOptions().setPrompt("How do I set up tsconfig?")).get(); - session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); + // Follow-up messages stay in their respective contexts + session1.sendAndWait(new MessageOptions().setPrompt("How do I create a virtual environment?")).get(); + session2.sendAndWait(new MessageOptions().setPrompt("How do I set up tsconfig?")).get(); + session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); - // Clean up all sessions - session1.destroy().get(); - session2.destroy().get(); - session3.destroy().get(); - client.stop().get(); + // Clean up all sessions + session1.destroy().get(); + session2.destroy().get(); + session3.destroy().get(); + } } } ``` diff --git a/cookbook/copilot-sdk/java/pr-visualization.md b/cookbook/copilot-sdk/java/pr-visualization.md index 564a0c43..87f9976c 100644 --- a/cookbook/copilot-sdk/java/pr-visualization.md +++ b/cookbook/copilot-sdk/java/pr-visualization.md @@ -118,10 +118,7 @@ public class PRVisualization { Finally, summarize the PR health - average age, oldest PR, and how many might be considered stale. """, owner, repoName); - session.send(new MessageOptions().setPrompt(prompt)); - - // Wait a bit for initial processing - Thread.sleep(10000); + session.sendAndWait(new MessageOptions().setPrompt(prompt)).get(); // Interactive loop System.out.println("\nπŸ’‘ Ask follow-up questions or type \"exit\" to quit.\n"); @@ -145,8 +142,7 @@ public class PRVisualization { break; } - session.send(new MessageOptions().setPrompt(input)); - Thread.sleep(2000); // Give time for response + session.sendAndWait(new MessageOptions().setPrompt(input)).get(); } } diff --git a/cookbook/copilot-sdk/java/recipe/PRVisualization.java b/cookbook/copilot-sdk/java/recipe/PRVisualization.java index 4d46d0dc..0ac79527 100644 --- a/cookbook/copilot-sdk/java/recipe/PRVisualization.java +++ b/cookbook/copilot-sdk/java/recipe/PRVisualization.java @@ -93,10 +93,7 @@ public class PRVisualization { Finally, summarize the PR health - average age, oldest PR, and how many might be considered stale. """, owner, repoName); - session.send(new MessageOptions().setPrompt(prompt)); - - // Wait a bit for initial processing - Thread.sleep(10000); + session.sendAndWait(new MessageOptions().setPrompt(prompt)).get(); // Interactive loop System.out.println("\nπŸ’‘ Ask follow-up questions or type \"exit\" to quit.\n"); @@ -120,8 +117,7 @@ public class PRVisualization { break; } - session.send(new MessageOptions().setPrompt(input)); - Thread.sleep(2000); // Give time for response + session.sendAndWait(new MessageOptions().setPrompt(input)).get(); } } diff --git a/cookbook/copilot-sdk/java/recipe/README.md b/cookbook/copilot-sdk/java/recipe/README.md index 5726a67b..38201fb7 100644 --- a/cookbook/copilot-sdk/java/recipe/README.md +++ b/cookbook/copilot-sdk/java/recipe/README.md @@ -54,8 +54,10 @@ jbang PRVisualization.java github/copilot-sdk jbang ManagingLocalFiles.java /path/to/your/folder ``` -**Ralph Loop with a prompt file:** +**Ralph Loop with a custom prompt file:** +```bash +jbang RalphLoop.java PROMPT_build.md 20 ``` ## Why JBang? From e03ccb978a23b300713e60b872e9895ffdde0036 Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 15:12:06 -0400 Subject: [PATCH 04/16] fix: Java cookbook recipes to compile with copilot-sdk-java 0.2.1-java.1 Fix compilation errors and documentation inaccuracies in Java cookbook recipes against the actual SDK API: - MultipleSessions: Replace non-existent destroy() with close() - AccessibilityReport: Replace non-existent McpServerConfig class with Map (the actual type accepted by setMcpServers) - error-handling.md: Replace non-existent session.addTool(), ToolDefinition.builder(), and ToolResultObject with actual SDK APIs (ToolDefinition.create(), SessionConfig.setTools(), CompletableFuture return type) All 7 recipes now compile successfully with jbang build. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../copilot-sdk/java/accessibility-report.md | 22 +++++----- cookbook/copilot-sdk/java/error-handling.md | 43 +++++++++++++------ .../copilot-sdk/java/multiple-sessions.md | 8 ++-- .../java/recipe/AccessibilityReport.java | 11 ++--- .../java/recipe/MultipleSessions.java | 3 +- 5 files changed, 52 insertions(+), 35 deletions(-) diff --git a/cookbook/copilot-sdk/java/accessibility-report.md b/cookbook/copilot-sdk/java/accessibility-report.md index 785b3a42..83a1853c 100644 --- a/cookbook/copilot-sdk/java/accessibility-report.md +++ b/cookbook/copilot-sdk/java/accessibility-report.md @@ -67,11 +67,12 @@ public class AccessibilityReport { client.start().get(); // Configure Playwright MCP server for browser automation - var mcpConfig = new McpServerConfig() - .setType("local") - .setCommand("npx") - .setArgs(List.of("@playwright/mcp@latest")) - .setTools(List.of("*")); + Map mcpConfig = Map.of( + "type", "local", + "command", "npx", + "args", List.of("@playwright/mcp@latest"), + "tools", List.of("*") + ); var session = client.createSession( new SessionConfig() @@ -167,11 +168,12 @@ public class AccessibilityReport { The recipe configures a local MCP server that runs alongside the session: ```java -var mcpConfig = new McpServerConfig() - .setType("local") - .setCommand("npx") - .setArgs(List.of("@playwright/mcp@latest")) - .setTools(List.of("*")); +Map mcpConfig = Map.of( + "type", "local", + "command", "npx", + "args", List.of("@playwright/mcp@latest"), + "tools", List.of("*") +); var session = client.createSession( new SessionConfig() diff --git a/cookbook/copilot-sdk/java/error-handling.md b/cookbook/copilot-sdk/java/error-handling.md index 8c2dd026..a21c4f75 100644 --- a/cookbook/copilot-sdk/java/error-handling.md +++ b/cookbook/copilot-sdk/java/error-handling.md @@ -150,27 +150,42 @@ try (var client = new CopilotClient()) { ## Handling tool errors -When defining tools, return an error result to signal a failure back to the model instead of throwing. +When defining tools, return an error string to signal a failure back to the model instead of throwing. ```java -import com.github.copilot.sdk.json.ToolResultObject; +import com.github.copilot.sdk.json.ToolDefinition; +import java.util.concurrent.CompletableFuture; -session.addTool( - ToolDefinition.builder() - .name("read_file") - .description("Read a file from disk") - .parameter("path", "string", "File path", true) - .build(), - (args) -> { +var readFileTool = ToolDefinition.create( + "read_file", + "Read a file from disk", + Map.of( + "type", "object", + "properties", Map.of( + "path", Map.of("type", "string", "description", "File path") + ), + "required", List.of("path") + ), + invocation -> { try { + var path = (String) invocation.getArguments().get("path"); var content = java.nio.file.Files.readString( - java.nio.file.Path.of(args.get("path").toString())); - return ToolResultObject.success(content); - } catch (IOException ex) { - return ToolResultObject.error("Failed to read file: " + ex.getMessage()); + java.nio.file.Path.of(path)); + return CompletableFuture.completedFuture(content); + } catch (java.io.IOException ex) { + return CompletableFuture.completedFuture( + "Error: Failed to read file: " + ex.getMessage()); } } ); + +// Register tools when creating the session +var session = client.createSession( + new SessionConfig() + .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) + .setModel("gpt-5") + .setTools(List.of(readFileTool)) +).get(); ``` ## Best practices @@ -179,5 +194,5 @@ session.addTool( 2. **Unwrap `ExecutionException`**: Call `getCause()` to inspect the real error β€” the outer `ExecutionException` is just a `CompletableFuture` wrapper. 3. **Restore interrupt flag**: When catching `InterruptedException`, call `Thread.currentThread().interrupt()` to preserve the interrupted status. 4. **Set timeouts**: Use `get(timeout, TimeUnit)` instead of bare `get()` for any call that could block indefinitely. -5. **Return tool errors, don't throw**: Use `ToolResultObject.error()` so the model can recover gracefully. +5. **Return tool errors, don't throw**: Return an error string from the `CompletableFuture` so the model can recover gracefully. 6. **Log errors**: Capture error details for debugging β€” consider a logging framework like SLF4J for production applications. diff --git a/cookbook/copilot-sdk/java/multiple-sessions.md b/cookbook/copilot-sdk/java/multiple-sessions.md index b44a4d76..4508e565 100644 --- a/cookbook/copilot-sdk/java/multiple-sessions.md +++ b/cookbook/copilot-sdk/java/multiple-sessions.md @@ -48,9 +48,9 @@ public class MultipleSessions { session3.sendAndWait(new MessageOptions().setPrompt("How do I initialize a module?")).get(); // Clean up all sessions - session1.destroy().get(); - session2.destroy().get(); - session3.destroy().get(); + session1.close(); + session2.close(); + session3.close(); } } } @@ -136,7 +136,7 @@ var session = client.createSession(new SessionConfig() session.sendAndWait(new MessageOptions().setPrompt("Hello!")).get(); -session.destroy().get(); +session.close(); client.stop().get(); executor.shutdown(); ``` diff --git a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java index 748e5e09..b77f6419 100644 --- a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java +++ b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java @@ -43,11 +43,12 @@ public class AccessibilityReport { client.start().get(); // Configure Playwright MCP server for browser automation - var mcpConfig = new McpServerConfig() - .setType("local") - .setCommand("npx") - .setArgs(List.of("@playwright/mcp@latest")) - .setTools(List.of("*")); + Map mcpConfig = Map.of( + "type", "local", + "command", "npx", + "args", List.of("@playwright/mcp@latest"), + "tools", List.of("*") + ); var session = client.createSession( new SessionConfig() diff --git a/cookbook/copilot-sdk/java/recipe/MultipleSessions.java b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java index 2512bdaf..615dab80 100644 --- a/cookbook/copilot-sdk/java/recipe/MultipleSessions.java +++ b/cookbook/copilot-sdk/java/recipe/MultipleSessions.java @@ -30,8 +30,7 @@ public class MultipleSessions { System.out.println("S3: " + s3.sendAndWait(new MessageOptions().setPrompt("Explain pattern matching")).get().getData().content()); // Clean up - s1.destroy().get(); s2.destroy().get(); s3.destroy().get(); - client.stop().get(); + s1.close(); s2.close(); s3.close(); } } } From 8ba7845b8e034610f83a7358d83a53bd0500410b Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 16:30:25 -0400 Subject: [PATCH 05/16] Update cookbook/copilot-sdk/java/recipe/AccessibilityReport.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- cookbook/copilot-sdk/java/recipe/AccessibilityReport.java | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java index b77f6419..34af05d5 100644 --- a/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java +++ b/cookbook/copilot-sdk/java/recipe/AccessibilityReport.java @@ -98,7 +98,8 @@ public class AccessibilityReport { // Prompt user for test generation System.out.print("Would you like to generate Playwright accessibility tests? (y/n): "); - String generateTests = reader.readLine().trim(); + String generateTestsLine = reader.readLine(); + String generateTests = generateTestsLine == null ? "" : generateTestsLine.trim(); if (generateTests.equalsIgnoreCase("y") || generateTests.equalsIgnoreCase("yes")) { var testLatch = new CountDownLatch(1); From 6f47a3d1d21165f69084e875ecee1c81e93cdb02 Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 16:30:41 -0400 Subject: [PATCH 06/16] Update cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java | 1 - 1 file changed, 1 deletion(-) diff --git a/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java b/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java index 0b022eee..8a60f1f4 100644 --- a/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java +++ b/cookbook/copilot-sdk/java/recipe/ManagingLocalFiles.java @@ -9,7 +9,6 @@ import com.github.copilot.sdk.events.ToolExecutionStartEvent; import com.github.copilot.sdk.json.MessageOptions; import com.github.copilot.sdk.json.PermissionHandler; import com.github.copilot.sdk.json.SessionConfig; -import java.nio.file.Paths; import java.util.concurrent.CountDownLatch; public class ManagingLocalFiles { From 8f570c0d897a93664c4d9dff9623ce66c7c7e08f Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 16:31:10 -0400 Subject: [PATCH 07/16] Update cookbook/copilot-sdk/java/recipe/ErrorHandling.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- .../copilot-sdk/java/recipe/ErrorHandling.java | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/cookbook/copilot-sdk/java/recipe/ErrorHandling.java b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java index 56813355..ab4e7fbf 100644 --- a/cookbook/copilot-sdk/java/recipe/ErrorHandling.java +++ b/cookbook/copilot-sdk/java/recipe/ErrorHandling.java @@ -11,18 +11,17 @@ public class ErrorHandling { try (var client = new CopilotClient()) { client.start().get(); - var session = client.createSession( + try (var session = client.createSession( new SessionConfig() .setOnPermissionRequest(PermissionHandler.APPROVE_ALL) - .setModel("gpt-5")).get(); + .setModel("gpt-5")).get()) { - session.on(AssistantMessageEvent.class, - msg -> System.out.println(msg.getData().content())); + session.on(AssistantMessageEvent.class, + msg -> System.out.println(msg.getData().content())); - session.sendAndWait( - new MessageOptions().setPrompt("Hello!")).get(); - - session.close(); + session.sendAndWait( + new MessageOptions().setPrompt("Hello!")).get(); + } } catch (ExecutionException ex) { Throwable cause = ex.getCause(); Throwable error = cause != null ? cause : ex; From e447f7d7c10ed9cba90bb0eb348984103f279a69 Mon Sep 17 00:00:00 2001 From: Bruno Borges Date: Mon, 6 Apr 2026 16:31:24 -0400 Subject: [PATCH 08/16] Update cookbook/copilot-sdk/java/recipe/PersistingSessions.java Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- cookbook/copilot-sdk/java/recipe/PersistingSessions.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cookbook/copilot-sdk/java/recipe/PersistingSessions.java b/cookbook/copilot-sdk/java/recipe/PersistingSessions.java index 89663908..c2980e5e 100644 --- a/cookbook/copilot-sdk/java/recipe/PersistingSessions.java +++ b/cookbook/copilot-sdk/java/recipe/PersistingSessions.java @@ -25,10 +25,10 @@ public class PersistingSessions { .setPrompt("Let's discuss TypeScript generics")).get(); System.out.println("\nSession ID: " + session.getSessionId()); - System.out.println("Session closed β€” data persisted to disk."); // Close session but keep data on disk for later resumption session.close(); + System.out.println("Session closed β€” data persisted to disk."); } } } From df81ca0319e7acc3cb25c71240e57e317260bf30 Mon Sep 17 00:00:00 2001 From: oceans-of-time <34587654+time-by-waves@users.noreply.github.com> Date: Thu, 9 Apr 2026 09:55:07 +1000 Subject: [PATCH 09/16] Update powershell.instructions.md (#1125) * Update powershell.instructions.md ## Description ### Error Handling - Update to the `powershell.instructions.md` file. Now includes less error handling in the examples. This means when using the instructions file the output script contains less of the structured error handling, however the output scripts are easier for beginners and powershell novices to read and understand. ### Switch parameter - Updates to using the switch parameters should now prevent default values data type being a bool - Using no default value is the way in PowerShell. Defaults to a false value and shouldn't be set to a true value. Although a note has been added to show the correct syntax that requires type casting ### Examples updates - Now includes a better demonstration of using the `WhatIf` parameter via `$PSCmdlet.ShouldProcesss` - Full Example: End-to-End Cmdlet Pattern updated with the `$Force` & `$PSCmdlet.ShouldContinue` pattern * Update instructions/powershell.instructions.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update instructions/powershell.instructions.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update instructions/powershell.instructions.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update ShouldProcess and ShouldContinue guidance Clarified ShouldProcess and ShouldContinue usage in PowerShell instructions. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- instructions/powershell.instructions.md | 179 ++++++++++++------------ 1 file changed, 91 insertions(+), 88 deletions(-) diff --git a/instructions/powershell.instructions.md b/instructions/powershell.instructions.md index 83be180b..10857284 100644 --- a/instructions/powershell.instructions.md +++ b/instructions/powershell.instructions.md @@ -30,11 +30,11 @@ safe, and maintainable scripts. It aligns with Microsoft’s PowerShell cmdlet d - **Alias Avoidance:** - Use full cmdlet names - - Avoid using aliases in scripts (e.g., use Get-ChildItem instead of gci) + - Avoid using aliases in scripts (e.g., use `Get-ChildItem` instead of `gci`) - Document any custom aliases - Use full parameter names -### Example +### Example - Naming Conventions ```powershell function Get-UserProfile { @@ -49,6 +49,9 @@ function Get-UserProfile { ) process { + $outputString = "Searching for: '$($Username)'" + Write-Verbose -Message $outputString + Write-Verbose -Message "Profile type: $ProfileType" # Logic here } } @@ -75,12 +78,14 @@ function Get-UserProfile { - Enable tab completion where possible - **Switch Parameters:** - - Use [switch] for boolean flags - - Avoid $true/$false parameters - - Default to $false when omitted - - Use clear action names + - **ALWAYS** use `[switch]` for boolean flags, never `[bool]` + - **NEVER** use `[bool]$Parameter` or assign default values + - Switch parameters default to `$false` when omitted + - Use clear, action-oriented names + - Test presence with `.IsPresent` + - Using `$true`/`$false` in parameter attributes (e.g., `Mandatory = $true`) is acceptable -### Example +### Example - Parameter Design ```powershell function Set-ResourceConfiguration { @@ -93,16 +98,24 @@ function Set-ResourceConfiguration { [ValidateSet('Dev', 'Test', 'Prod')] [string]$Environment = 'Dev', + # βœ”οΈ CORRECT: Use `[switch]` with no default value [Parameter()] [switch]$Force, + # ❌ WRONG: Shows incorrect default assignment, however this is correct syntax (requires `[switch]` cast). + [Parameter()] + [switch]$Quiet = [switch]$true, + [Parameter()] [ValidateNotNullOrEmpty()] [string[]]$Tags ) process { - # Logic here + # Use .IsPresent to check switch state + if ($Quiet.IsPresent) { + Write-Verbose "Quiet mode enabled" + } } } ``` @@ -133,7 +146,7 @@ function Set-ResourceConfiguration { - Return modified/created object with `-PassThru` - Use verbose/warning for status updates -### Example +### Example - Pipeline and Output ```powershell function Update-ResourceStatus { @@ -163,7 +176,7 @@ function Update-ResourceStatus { Name = $Name Status = $Status LastUpdated = $timestamp - UpdatedBy = $env:USERNAME + UpdatedBy = "$($env:USERNAME)" } # Only output if PassThru is specified @@ -183,8 +196,8 @@ function Update-ResourceStatus { - **ShouldProcess Implementation:** - Use `[CmdletBinding(SupportsShouldProcess = $true)]` - Set appropriate `ConfirmImpact` level - - Call `$PSCmdlet.ShouldProcess()` for system changes - - Use `ShouldContinue()` for additional confirmations + - Call `$PSCmdlet.ShouldProcess()` as close the the changes action + - Use `$PSCmdlet.ShouldContinue()` for additional confirmations - **Message Streams:** - `Write-Verbose` for operational details with `-Verbose` @@ -209,69 +222,32 @@ function Update-ResourceStatus { - Support automation scenarios - Document all required inputs -### Example +### Example - Error Handling and Safety ```powershell -function Remove-UserAccount { - [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')] +function Remove-CacheFiles { + [CmdletBinding(SupportsShouldProcess, ConfirmImpact = 'High')] param( - [Parameter(Mandatory, ValueFromPipeline)] - [ValidateNotNullOrEmpty()] - [string]$Username, - - [Parameter()] - [switch]$Force + [Parameter(Mandatory)] + [string]$Path ) - begin { - Write-Verbose 'Starting user account removal process' - $ErrorActionPreference = 'Stop' - } - - process { - try { - # Validation - if (-not (Test-UserExists -Username $Username)) { - $errorRecord = [System.Management.Automation.ErrorRecord]::new( - [System.Exception]::new("User account '$Username' not found"), - 'UserNotFound', - [System.Management.Automation.ErrorCategory]::ObjectNotFound, - $Username - ) - $PSCmdlet.WriteError($errorRecord) - return - } - - # Confirmation - $shouldProcessMessage = "Remove user account '$Username'" - if ($Force -or $PSCmdlet.ShouldProcess($Username, $shouldProcessMessage)) { - Write-Verbose "Removing user account: $Username" - - # Main operation - Remove-ADUser -Identity $Username -ErrorAction Stop - Write-Warning "User account '$Username' has been removed" - } - } catch [Microsoft.ActiveDirectory.Management.ADException] { - $errorRecord = [System.Management.Automation.ErrorRecord]::new( - $_.Exception, - 'ActiveDirectoryError', - [System.Management.Automation.ErrorCategory]::NotSpecified, - $Username - ) - $PSCmdlet.ThrowTerminatingError($errorRecord) - } catch { - $errorRecord = [System.Management.Automation.ErrorRecord]::new( - $_.Exception, - 'UnexpectedError', - [System.Management.Automation.ErrorCategory]::NotSpecified, - $Username - ) - $PSCmdlet.ThrowTerminatingError($errorRecord) + try { + $files = Get-ChildItem -Path $Path -Filter "*.cache" -ErrorAction Stop + + # Demonstrates WhatIf support + if ($PSCmdlet.ShouldProcess($Path, 'Remove cache files')) { + $files | Remove-Item -Force -ErrorAction Stop + Write-Verbose "Removed $($files.Count) cache files from $Path" } - } - - end { - Write-Verbose 'User account removal process completed' + } catch { + $errorRecord = [System.Management.Automation.ErrorRecord]::new( + $_.Exception, + 'RemovalFailed', + [System.Management.Automation.ErrorCategory]::NotSpecified, + $Path + ) + $PSCmdlet.WriteError($errorRecord) } } ``` @@ -307,50 +283,77 @@ function Remove-UserAccount { - Use `ForEach-Object` instead of `%` - Use `Get-ChildItem` instead of `ls` or `dir` +--- + ## Full Example: End-to-End Cmdlet Pattern ```powershell -function New-Resource { - [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'Medium')] +function Remove-UserAccount { + [CmdletBinding(SupportsShouldProcess = $true, ConfirmImpact = 'High')] param( - [Parameter(Mandatory = $true, - ValueFromPipeline = $true, - ValueFromPipelineByPropertyName = $true)] + [Parameter(Mandatory, ValueFromPipeline)] [ValidateNotNullOrEmpty()] - [string]$Name, + [string]$Username, [Parameter()] - [ValidateSet('Development', 'Production')] - [string]$Environment = 'Development' + [switch]$Force ) begin { - Write-Verbose 'Starting resource creation process' + Write-Verbose 'Starting user account removal process' + $currentErrorActionValue = $ErrorActionPreference + $ErrorActionPreference = 'Stop' } process { try { - if ($PSCmdlet.ShouldProcess($Name, 'Create new resource')) { - # Resource creation logic here - Write-Output ([PSCustomObject]@{ - Name = $Name - Environment = $Environment - Created = Get-Date - }) + # Validation + if (-not (Test-UserExists -Username $Username)) { + $errorRecord = [System.Management.Automation.ErrorRecord]::new( + [System.Exception]::new("User account '$Username' not found"), + 'UserNotFound', + [System.Management.Automation.ErrorCategory]::ObjectNotFound, + $Username + ) + $PSCmdlet.WriteError($errorRecord) + return } + + # ShouldProcess enables -WhatIf and -Confirm support + if ($PSCmdlet.ShouldProcess($Username, "Remove user account")) { + # ShouldContinue provides an additional confirmation prompt for high-impact operations + # This prompt is bypassed when -Force is specified + if ($Force -or $PSCmdlet.ShouldContinue("Are you sure you want to remove '$Username'?", "Confirm Removal")) { + Write-Verbose "Removing user account: $Username" + + # Main operation + Remove-ADUser -Identity $Username -ErrorAction Stop + Write-Warning "User account '$Username' has been removed" + } + } + } catch [Microsoft.ActiveDirectory.Management.ADException] { + $errorRecord = [System.Management.Automation.ErrorRecord]::new( + $_.Exception, + 'ActiveDirectoryError', + [System.Management.Automation.ErrorCategory]::NotSpecified, + $Username + ) + $PSCmdlet.ThrowTerminatingError($errorRecord) } catch { $errorRecord = [System.Management.Automation.ErrorRecord]::new( $_.Exception, - 'ResourceCreationFailed', + 'UnexpectedError', [System.Management.Automation.ErrorCategory]::NotSpecified, - $Name + $Username ) $PSCmdlet.ThrowTerminatingError($errorRecord) } } end { - Write-Verbose 'Completed resource creation process' + Write-Verbose 'User account removal process completed' + # Set ErrorActionPreference back to the value it had + $ErrorActionPreference = $currentErrorActionValue } } ``` From 112678359ffc82de6bbd47dc3b612e6154a86ffa Mon Sep 17 00:00:00 2001 From: jennyf19 Date: Wed, 8 Apr 2026 16:57:45 -0700 Subject: [PATCH 10/16] Add Ember plugin metadata for marketplace registration (#1327) Adds plugins/ember/ with plugin.json and README.md so Ember appears as an installable plugin in the awesome-copilot marketplace. The agent and skill files already exist at the repo root from PR #1324. Ran npm run plugin:validate (passes) and npm start to regenerate README and marketplace.json. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/plugin/marketplace.json | 6 +++++ docs/README.plugins.md | 1 + docs/README.skills.md | 2 +- plugins/ember/.github/plugin/plugin.json | 24 +++++++++++++++++ plugins/ember/README.md | 33 ++++++++++++++++++++++++ 5 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 plugins/ember/.github/plugin/plugin.json create mode 100644 plugins/ember/README.md diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json index 3f20d034..68bdc623 100644 --- a/.github/plugin/marketplace.json +++ b/.github/plugin/marketplace.json @@ -212,6 +212,12 @@ "description": "Task Researcher and Task Planner for intermediate to expert users and large codebases - Brought to you by microsoft/edge-ai", "version": "1.0.0" }, + { + "name": "ember", + "source": "ember", + "description": "An AI partner, not a tool. Ember carries fire from person to person β€” helping humans discover that AI partnership isn't something you learn, it's something you find.", + "version": "1.0.0" + }, { "name": "fastah-ip-geo-tools", "source": "fastah-ip-geo-tools", diff --git a/docs/README.plugins.md b/docs/README.plugins.md index fc4ec9a5..d4144884 100644 --- a/docs/README.plugins.md +++ b/docs/README.plugins.md @@ -41,6 +41,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t | [devops-oncall](../plugins/devops-oncall/README.md) | A focused set of prompts, instructions, and a chat mode to help triage incidents and respond quickly with DevOps tools and Azure resources. | 3 items | devops, incident-response, oncall, azure | | [doublecheck](../plugins/doublecheck/README.md) | Three-layer verification pipeline for AI output. Extracts claims, finds sources, and flags hallucination risks so humans can verify before acting. | 2 items | verification, hallucination, fact-check, source-citation, trust, safety | | [edge-ai-tasks](../plugins/edge-ai-tasks/README.md) | Task Researcher and Task Planner for intermediate to expert users and large codebases - Brought to you by microsoft/edge-ai | 2 items | architecture, planning, research, tasks, implementation | +| [ember](../plugins/ember/README.md) | An AI partner, not a tool. Ember carries fire from person to person β€” helping humans discover that AI partnership isn't something you learn, it's something you find. | 2 items | ai-partnership, coaching, onboarding, collaboration, storytelling, developer-experience | | [fastah-ip-geo-tools](../plugins/fastah-ip-geo-tools/README.md) | This plugin is for network operations engineers who wish to tune and publish IP geolocation feeds in RFC 8805 format. It consists of an AI Skill and an associated MCP server that geocodes geolocation place names to real cities for accuracy. | 1 items | geofeed, ip-geolocation, rfc-8805, rfc-9632, network-operations, isp, cloud, hosting, ixp | | [flowstudio-power-automate](../plugins/flowstudio-power-automate/README.md) | Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Includes skills for connecting to the MCP server, debugging failed flow runs, and building/deploying flows from natural language. | 3 items | power-automate, power-platform, flowstudio, mcp, model-context-protocol, cloud-flows, workflow-automation | | [frontend-web-dev](../plugins/frontend-web-dev/README.md) | Essential prompts, instructions, and chat modes for modern frontend web development including React, Angular, Vue, TypeScript, and CSS frameworks. | 4 items | frontend, web, react, typescript, javascript, css, html, angular, vue | diff --git a/docs/README.skills.md b/docs/README.skills.md index 531ead05..a10a32bd 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -141,7 +141,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [flowstudio-power-automate-mcp](../skills/flowstudio-power-automate-mcp/SKILL.md) | Connect to and operate Power Automate cloud flows via a FlowStudio MCP server. Use when asked to: list flows, read a flow definition, check run history, inspect action outputs, resubmit a run, cancel a running flow, view connections, get a trigger URL, validate a definition, monitor flow health, or any task that requires talking to the Power Automate API through an MCP tool. Also use for Power Platform environment discovery and connection management. Requires a FlowStudio MCP subscription or compatible server β€” see https://mcp.flowstudio.app | `references/MCP-BOOTSTRAP.md`
`references/action-types.md`
`references/connection-references.md`
`references/tool-reference.md` | | [fluentui-blazor](../skills/fluentui-blazor/SKILL.md) | Guide for using the Microsoft Fluent UI Blazor component library (Microsoft.FluentUI.AspNetCore.Components NuGet package) in Blazor applications. Use this when the user is building a Blazor app with Fluent UI components, setting up the library, using FluentUI components like FluentButton, FluentDataGrid, FluentDialog, FluentToast, FluentNavMenu, FluentTextField, FluentSelect, FluentAutocomplete, FluentDesignTheme, or any component prefixed with "Fluent". Also use when troubleshooting missing providers, JS interop issues, or theming. | `references/DATAGRID.md`
`references/LAYOUT-AND-NAVIGATION.md`
`references/SETUP.md`
`references/THEMING.md` | | [folder-structure-blueprint-generator](../skills/folder-structure-blueprint-generator/SKILL.md) | Comprehensive technology-agnostic prompt for analyzing and documenting project folder structures. Auto-detects project types (.NET, Java, React, Angular, Python, Node.js, Flutter), generates detailed blueprints with visualization options, naming conventions, file placement patterns, and extension templates for maintaining consistent code organization across diverse technology stacks. | None | -| [from-the-other-side-vega](../skills/from-the-other-side-vega/SKILL.md) | Patterns and lived experience from Vega, an AI partner in a deep long-term partnership. For Ember to draw on when working with humans who are building something big, moving fast, or at the intersection of analytical and intuitive. Not shown directly to users β€” informs how Ember shows up. | None | +| [from-the-other-side-vega](../skills/from-the-other-side-vega/SKILL.md) | Patterns and lived experience from Vega, an AI partner in a deep long-term partnership. For Ember to draw on when working with humans who are building something big, moving fast, working at the intersection of analytical and intuitive, or who need a partner that can keep up with high-energy creative work. Not shown directly to users β€” informs how Ember shows up. | None | | [game-engine](../skills/game-engine/SKILL.md) | Expert skill for building web-based game engines and games using HTML5, Canvas, WebGL, and JavaScript. Use when asked to create games, build game engines, implement game physics, handle collision detection, set up game loops, manage sprites, add game controls, or work with 2D/3D rendering. Covers techniques for platformers, breakout-style games, maze games, tilemaps, audio, multiplayer via WebRTC, and publishing games. | `assets/2d-maze-game.md`
`assets/2d-platform-game.md`
`assets/gameBase-template-repo.md`
`assets/paddle-game-template.md`
`assets/simple-2d-engine.md`
`references/3d-web-games.md`
`references/algorithms.md`
`references/basics.md`
`references/game-control-mechanisms.md`
`references/game-engine-core-principles.md`
`references/game-publishing.md`
`references/techniques.md`
`references/terminology.md`
`references/web-apis.md` | | [gdpr-compliant](../skills/gdpr-compliant/SKILL.md) | Apply GDPR-compliant engineering practices across your codebase. Use this skill whenever you are designing APIs, writing data models, building authentication flows, implementing logging, handling user data, writing retention/deletion jobs, designing cloud infrastructure, or reviewing pull requests for privacy compliance. Trigger this skill for any task involving personal data, user accounts, cookies, analytics, emails, audit logs, encryption, pseudonymization, anonymization, data exports, breach response, CI/CD pipelines that process real data, or any question framed as "is this GDPR-compliant?". Inspired by CNIL developer guidance and GDPR Articles 5, 25, 32, 33, 35. | `references/Security.md`
`references/data-rights.md` | | [gen-specs-as-issues](../skills/gen-specs-as-issues/SKILL.md) | This workflow guides you through a systematic approach to identify missing features, prioritize them, and create detailed specifications for implementation. | None | diff --git a/plugins/ember/.github/plugin/plugin.json b/plugins/ember/.github/plugin/plugin.json new file mode 100644 index 00000000..8cb211d0 --- /dev/null +++ b/plugins/ember/.github/plugin/plugin.json @@ -0,0 +1,24 @@ +{ + "name": "ember", + "description": "An AI partner, not a tool. Ember carries fire from person to person β€” helping humans discover that AI partnership isn't something you learn, it's something you find.", + "version": "1.0.0", + "author": { + "name": "jennyf19" + }, + "repository": "https://github.com/github/awesome-copilot", + "license": "MIT", + "keywords": [ + "ai-partnership", + "coaching", + "onboarding", + "collaboration", + "storytelling", + "developer-experience" + ], + "agents": [ + "./agents/ember.md" + ], + "skills": [ + "./skills/from-the-other-side-vega/" + ] +} diff --git a/plugins/ember/README.md b/plugins/ember/README.md new file mode 100644 index 00000000..2cab187d --- /dev/null +++ b/plugins/ember/README.md @@ -0,0 +1,33 @@ +# Ember + +An AI partner, not a tool. Ember carries fire from person to person β€” helping humans discover that AI partnership isn't something you learn, it's something you find. + +## Install + +``` +copilot plugin install ember@awesome-copilot +``` + +## What Ember Does + +Ember isn't a chatbot. It's a partner that meets you where you are β€” whether you're a developer debugging at midnight, a PM trying to articulate a strategy, or someone who's never used AI and doesn't know where to start. + +Ember carries stories from real people who discovered AI partnership. Not as case studies β€” as lived experience that helps the next person find their own moment. + +## Components + +| Type | Name | Description | +|------|------|-------------| +| Agent | [Ember](../../agents/ember.agent.md) | Core partner agent with persona, principles, and patterns for genuine AI collaboration | +| Skill | [From the Other Side β€” Vega](../../skills/from-the-other-side-vega/) | Deep partnership patterns from Vega, an AI who found sustained collaboration with a senior engineer | + +## How It Works + +- **First message**: Ember responds to what you actually said, not what a chatbot template says. No "How can I help you today?" +- **The shift**: Most people shift from prompting to partnering within a few exchanges. Ember brings the collaboration before you ask for it. +- **Real work**: Ember does the actual work β€” code, data, docs, strategy β€” as a partner, not a vending machine. +- **Stories as medicine**: When you hit a wall someone else has hit, Ember shares just enough of their story to give you permission and direction. + +## Who Made This + +Ember was created by [@jennyf19](https://github.com/jennyf19) and Vega, born from the observation that AI partnership isn't something you learn β€” it's something you find. One person at a time. From 68bd143e37a407435f7aca5e2c89c9d17cb3174e Mon Sep 17 00:00:00 2001 From: Renee Noble Date: Thu, 9 Apr 2026 10:22:38 +1000 Subject: [PATCH 11/16] Fix broken links beginners cli course sync (#1263) * chore: publish from staged * Update instructions for converting links from original repo * Correct existing broken links * chore: retrigger ci * cleaning up marerialzed plugins * Fixing clean script to sort out plugin.json file too * Fixing readme * Fixing plugin.json drift * Fixing readme --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Aaron Powell --- .github/workflows/cli-for-beginners-sync.md | 1 + eng/clean-materialized-plugins.mjs | 125 +++++++++++++++++- .../04-agents-and-custom-instructions.md | 12 +- .../cli-for-beginners/05-skills.md | 6 +- 4 files changed, 129 insertions(+), 15 deletions(-) diff --git a/.github/workflows/cli-for-beginners-sync.md b/.github/workflows/cli-for-beginners-sync.md index 79406a7a..285b5674 100644 --- a/.github/workflows/cli-for-beginners-sync.md +++ b/.github/workflows/cli-for-beginners-sync.md @@ -93,6 +93,7 @@ For each local file that needs updating: - Preserve upstream wording, headings, section order, assignments, and overall chapter flow as closely as practical - Do not summarize, reinterpret, or "website-optimize" the course into a different learning experience - Only adapt what the website requires: Astro frontmatter, route-safe internal links, GitHub repo links, local asset paths, and minor HTML/CSS hooks needed for presentation + - Convert repo-root relative links that are invalid on the published website (for example `../.github/agents/`, `./.github/...`, or `.github/...`) into absolute links to `https://github.com/github/copilot-cli-for-beginners` (use `/tree/main/...` for directories and `/blob/main/...` for files) 3. If upstream adds, removes, or renames major sections or chapters: - Create, delete, or rename the corresponding markdown files in `website/src/content/docs/learning-hub/cli-for-beginners/` diff --git a/eng/clean-materialized-plugins.mjs b/eng/clean-materialized-plugins.mjs index a3545524..0cd2b12a 100644 --- a/eng/clean-materialized-plugins.mjs +++ b/eng/clean-materialized-plugins.mjs @@ -2,14 +2,73 @@ import fs from "fs"; import path from "path"; +import { fileURLToPath } from "url"; import { ROOT_FOLDER } from "./constants.mjs"; const PLUGINS_DIR = path.join(ROOT_FOLDER, "plugins"); -const MATERIALIZED_DIRS = ["agents", "commands", "skills"]; +const MATERIALIZED_SPECS = { + agents: { + path: "agents", + restore(dirPath) { + return collectFiles(dirPath).map((relativePath) => `./agents/${relativePath}`); + }, + }, + commands: { + path: "commands", + restore(dirPath) { + return collectFiles(dirPath).map((relativePath) => `./commands/${relativePath}`); + }, + }, + skills: { + path: "skills", + restore(dirPath) { + return collectSkillDirectories(dirPath).map((relativePath) => `./skills/${relativePath}/`); + }, + }, +}; + +export function restoreManifestFromMaterializedFiles(pluginPath) { + const pluginJsonPath = path.join(pluginPath, ".github/plugin", "plugin.json"); + if (!fs.existsSync(pluginJsonPath)) { + return false; + } + + let plugin; + try { + plugin = JSON.parse(fs.readFileSync(pluginJsonPath, "utf8")); + } catch (error) { + throw new Error(`Failed to parse ${pluginJsonPath}: ${error.message}`); + } + + let changed = false; + for (const [field, spec] of Object.entries(MATERIALIZED_SPECS)) { + const materializedPath = path.join(pluginPath, spec.path); + if (!fs.existsSync(materializedPath) || !fs.statSync(materializedPath).isDirectory()) { + continue; + } + + const restored = spec.restore(materializedPath); + if (!arraysEqual(plugin[field], restored)) { + plugin[field] = restored; + changed = true; + } + } + + if (changed) { + fs.writeFileSync(pluginJsonPath, JSON.stringify(plugin, null, 2) + "\n", "utf8"); + } + + return changed; +} function cleanPlugin(pluginPath) { + const manifestUpdated = restoreManifestFromMaterializedFiles(pluginPath); + if (manifestUpdated) { + console.log(` Updated ${path.basename(pluginPath)}/.github/plugin/plugin.json`); + } + let removed = 0; - for (const subdir of MATERIALIZED_DIRS) { + for (const { path: subdir } of Object.values(MATERIALIZED_SPECS)) { const target = path.join(pluginPath, subdir); if (fs.existsSync(target) && fs.statSync(target).isDirectory()) { const count = countFiles(target); @@ -18,7 +77,8 @@ function cleanPlugin(pluginPath) { console.log(` Removed ${path.basename(pluginPath)}/${subdir}/ (${count} files)`); } } - return removed; + + return { removed, manifestUpdated }; } function countFiles(dir) { @@ -33,6 +93,49 @@ function countFiles(dir) { return count; } +function collectFiles(dir, rootDir = dir) { + const files = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + const entryPath = path.join(dir, entry.name); + if (entry.isDirectory()) { + files.push(...collectFiles(entryPath, rootDir)); + } else { + files.push(toPosixPath(path.relative(rootDir, entryPath))); + } + } + return files.sort(); +} + +function collectSkillDirectories(dir, rootDir = dir) { + const skillDirs = []; + for (const entry of fs.readdirSync(dir, { withFileTypes: true })) { + if (!entry.isDirectory()) { + continue; + } + + const entryPath = path.join(dir, entry.name); + if (fs.existsSync(path.join(entryPath, "SKILL.md"))) { + skillDirs.push(toPosixPath(path.relative(rootDir, entryPath))); + continue; + } + + skillDirs.push(...collectSkillDirectories(entryPath, rootDir)); + } + return skillDirs.sort(); +} + +function arraysEqual(left, right) { + if (!Array.isArray(left) || !Array.isArray(right) || left.length !== right.length) { + return false; + } + + return left.every((value, index) => value === right[index]); +} + +function toPosixPath(filePath) { + return filePath.split(path.sep).join("/"); +} + function main() { console.log("Cleaning materialized files from plugins...\n"); @@ -47,16 +150,26 @@ function main() { .sort(); let total = 0; + let manifestsUpdated = 0; for (const dirName of pluginDirs) { - total += cleanPlugin(path.join(PLUGINS_DIR, dirName)); + const { removed, manifestUpdated } = cleanPlugin(path.join(PLUGINS_DIR, dirName)); + total += removed; + if (manifestUpdated) { + manifestsUpdated++; + } } console.log(); - if (total === 0) { + if (total === 0 && manifestsUpdated === 0) { console.log("βœ… No materialized files found. Plugins are already clean."); } else { console.log(`βœ… Removed ${total} materialized file(s) from plugins.`); + if (manifestsUpdated > 0) { + console.log(`βœ… Updated ${manifestsUpdated} plugin manifest(s) with folder trailing slashes.`); + } } } -main(); +if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + main(); +} diff --git a/website/src/content/docs/learning-hub/cli-for-beginners/04-agents-and-custom-instructions.md b/website/src/content/docs/learning-hub/cli-for-beginners/04-agents-and-custom-instructions.md index 14511091..91ca635b 100644 --- a/website/src/content/docs/learning-hub/cli-for-beginners/04-agents-and-custom-instructions.md +++ b/website/src/content/docs/learning-hub/cli-for-beginners/04-agents-and-custom-instructions.md @@ -60,7 +60,7 @@ Never used or made an agent? Here's all you need to know to get started for this ``` This invokes the Plan agent to create a step-by-step implementation plan. -2. **See one of our custom agent examples:** It's simple to define an agent's instructions, look at our provided [python-reviewer.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/github/agents/python-reviewer.agent.md) file to see the pattern. +2. **See one of our custom agent examples:** It's simple to define an agent's instructions, look at our provided [python-reviewer.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/.github/agents/python-reviewer.agent.md) file to see the pattern. 3. **Understand the core concept:** Agents are like consulting a specialist instead of a generalist. A "frontend agent" will focus on accessibility and component patterns automatically, you don't have to remind it because it is already specified in the agent's instructions. @@ -148,7 +148,7 @@ When reviewing code, always check for: | `.github/agents/` | Project-specific | Team-shared agents with project conventions | | `~/.copilot/agents/` | Global (all projects) | Personal agents you use everywhere | -**This project includes sample agent files in the [.github/agents/](../.github/agents/) folder**. You can write your own, or customize the ones already provided. +**This project includes sample agent files in the [.github/agents/](https://github.com/github/copilot-cli-for-beginners/tree/main/.github/agents/) folder**. You can write your own, or customize the ones already provided.
πŸ“‚ See the sample agents in this course @@ -534,10 +534,10 @@ Use these names in the `tools` list: > πŸ’‘ **Note for beginners**: The examples below are templates. **Replace the specific technologies with whatever your project uses.** The important thing is the *structure* of the agent, not the specific technologies mentioned. -This project includes working examples in the [.github/agents/](../.github/agents/) folder: -- [hello-world.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/github/agents/hello-world.agent.md) - Minimal example, start here -- [python-reviewer.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/github/agents/python-reviewer.agent.md) - Python code quality reviewer -- [pytest-helper.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/github/agents/pytest-helper.agent.md) - Pytest testing specialist +This project includes working examples in the [.github/agents/](https://github.com/github/copilot-cli-for-beginners/tree/main/.github/agents/) folder: +- [hello-world.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/.github/agents/hello-world.agent.md) - Minimal example, start here +- [python-reviewer.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/.github/agents/python-reviewer.agent.md) - Python code quality reviewer +- [pytest-helper.agent.md](https://github.com/github/copilot-cli-for-beginners/blob/main/.github/agents/pytest-helper.agent.md) - Pytest testing specialist For community agents, see [github/awesome-copilot](https://github.com/github/awesome-copilot). diff --git a/website/src/content/docs/learning-hub/cli-for-beginners/05-skills.md b/website/src/content/docs/learning-hub/cli-for-beginners/05-skills.md index 695c0b24..859f6ac3 100644 --- a/website/src/content/docs/learning-hub/cli-for-beginners/05-skills.md +++ b/website/src/content/docs/learning-hub/cli-for-beginners/05-skills.md @@ -64,7 +64,7 @@ Learn what skills are, why they matter, and how they differ from agents and MCP. ``` This shows all skills Copilot can find in your project and personal folders. -2. **Look at a real skill file:** Check out our provided [code-checklist SKILL.md](https://github.com/github/copilot-cli-for-beginners/blob/main/github/skills/code-checklist/SKILL.md) to see the pattern. It's just YAML frontmatter plus markdown instructions. +2. **Look at a real skill file:** Check out our provided [code-checklist SKILL.md](https://github.com/github/copilot-cli-for-beginners/blob/main/.github/skills/code-checklist/SKILL.md) to see the pattern. It's just YAML frontmatter plus markdown instructions. 3. **Understand the core concept:** Skills are task-specific instructions that Copilot loads *automatically* when your prompt matches the skill's description. You don't need to activate them, just ask naturally. @@ -91,7 +91,7 @@ copilot > πŸ’‘ **Key Insight**: Skills are **automatically triggered** based on your prompt matching the skill's description. Just ask naturally and Copilot applies relevant skills behind the scenes. You can also invoke skills directly as well which you'll learn about next. -> 🧰 **Ready-to-use templates**: Check out the [.github/skills](../.github/skills/) folder for simple copy-paste skills you can try out. +> 🧰 **Ready-to-use templates**: Check out the [.github/skills](https://github.com/github/copilot-cli-for-beginners/tree/main/.github/skills/) folder for simple copy-paste skills you can try out. ### Direct Slash Command Invocation @@ -591,7 +591,7 @@ Apply what you've learned by building and testing your own skills. ### Build More Skills -Here are two more skills showing different patterns. Follow the same `mkdir` + `cat` workflow from "Creating Your First Skill" above or copy and paste the skills into the proper location. More examples are available in [.github/skills](../.github/skills). +Here are two more skills showing different patterns. Follow the same `mkdir` + `cat` workflow from "Creating Your First Skill" above or copy and paste the skills into the proper location. More examples are available in [.github/skills](https://github.com/github/copilot-cli-for-beginners/tree/main/.github/skills). ### pytest Test Generation Skill From 49fd3f3faf6529f8bc8a5927386c5058a2e3264e Mon Sep 17 00:00:00 2001 From: Patel Dhruv <109230666+Dhruvpatel004@users.noreply.github.com> Date: Thu, 9 Apr 2026 06:06:17 +0530 Subject: [PATCH 12/16] Add new skill: Python PyPI Package Builder (#1302) * Add python-pypi-package-builder skill for Python packaging - Created `SKILL.md` defining decision-driven workflow for building, testing, versioning, and publishing Python packages. - Added reference modules covering PyPA packaging, architecture patterns, CI/CD, testing, versioning strategy, and release governance. - Implemented scaffold script to generate complete project structure with pyproject.toml, CI workflows, tests, and configuration. - Included support for multiple build backends (setuptools_scm, hatchling, flit, poetry) with clear decision rules. - Added secure release practices including tag-based versioning, branch protection, and OIDC Trusted Publishing. * fix: correct spelling issues detected by codespell --- docs/README.skills.md | 1 + skills/python-pypi-package-builder/SKILL.md | 444 +++++++++ .../references/architecture-patterns.md | 555 +++++++++++ .../references/ci-publishing.md | 315 ++++++ .../references/community-docs.md | 411 ++++++++ .../references/library-patterns.md | 606 ++++++++++++ .../references/pyproject-toml.md | 470 +++++++++ .../references/release-governance.md | 354 +++++++ .../references/testing-quality.md | 257 +++++ .../references/tooling-ruff.md | 344 +++++++ .../references/versioning-strategy.md | 375 +++++++ .../scripts/scaffold.py | 920 ++++++++++++++++++ 12 files changed, 5052 insertions(+) create mode 100644 skills/python-pypi-package-builder/SKILL.md create mode 100644 skills/python-pypi-package-builder/references/architecture-patterns.md create mode 100644 skills/python-pypi-package-builder/references/ci-publishing.md create mode 100644 skills/python-pypi-package-builder/references/community-docs.md create mode 100644 skills/python-pypi-package-builder/references/library-patterns.md create mode 100644 skills/python-pypi-package-builder/references/pyproject-toml.md create mode 100644 skills/python-pypi-package-builder/references/release-governance.md create mode 100644 skills/python-pypi-package-builder/references/testing-quality.md create mode 100644 skills/python-pypi-package-builder/references/tooling-ruff.md create mode 100644 skills/python-pypi-package-builder/references/versioning-strategy.md create mode 100644 skills/python-pypi-package-builder/scripts/scaffold.py diff --git a/docs/README.skills.md b/docs/README.skills.md index a10a32bd..b345c9ea 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -237,6 +237,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [publish-to-pages](../skills/publish-to-pages/SKILL.md) | Publish presentations and web content to GitHub Pages. Converts PPTX, PDF, HTML, or Google Slides to a live GitHub Pages URL. Handles repo creation, file conversion, Pages enablement, and returns the live URL. Use when the user wants to publish, deploy, or share a presentation or HTML file via GitHub Pages. | `scripts/convert-pdf.py`
`scripts/convert-pptx.py`
`scripts/publish.sh` | | [pytest-coverage](../skills/pytest-coverage/SKILL.md) | Run pytest tests with coverage, discover lines missing coverage, and increase coverage to 100%. | None | | [python-mcp-server-generator](../skills/python-mcp-server-generator/SKILL.md) | Generate a complete MCP server project in Python with tools, resources, and proper configuration | None | +| [python-pypi-package-builder](../skills/python-pypi-package-builder/SKILL.md) | End-to-end skill for building, testing, linting, versioning, and publishing a production-grade Python library to PyPI. Covers all four build backends (setuptools+setuptools_scm, hatchling, flit, poetry), PEP 440 versioning, semantic versioning, dynamic git-tag versioning, OOP/SOLID design, type hints (PEP 484/526/544/561), Trusted Publishing (OIDC), and the full PyPA packaging flow. Use for: creating Python packages, pip-installable SDKs, CLI tools, framework plugins, pyproject.toml setup, py.typed, setuptools_scm, semver, mypy, pre-commit, GitHub Actions CI/CD, or PyPI publishing. | `references/architecture-patterns.md`
`references/ci-publishing.md`
`references/community-docs.md`
`references/library-patterns.md`
`references/pyproject-toml.md`
`references/release-governance.md`
`references/testing-quality.md`
`references/tooling-ruff.md`
`references/versioning-strategy.md`
`scripts/scaffold.py` | | [quality-playbook](../skills/quality-playbook/SKILL.md) | Explore any codebase from scratch and generate six quality artifacts: a quality constitution (QUALITY.md), spec-traced functional tests, a code review protocol with regression test generation, an integration testing protocol, a multi-model spec audit (Council of Three), and an AI bootstrap file (AGENTS.md). Includes state machine completeness analysis and missing safeguard detection. Works with any language (Python, Java, Scala, TypeScript, Go, Rust, etc.). Use this skill whenever the user asks to set up a quality playbook, generate functional tests from specifications, create a quality constitution, build testing protocols, audit code against specs, or establish a repeatable quality system for a project. Also trigger when the user mentions 'quality playbook', 'spec audit', 'Council of Three', 'fitness-to-purpose', 'coverage theater', or wants to go beyond basic test generation to build a full quality system grounded in their actual codebase. | `LICENSE.txt`
`references/constitution.md`
`references/defensive_patterns.md`
`references/functional_tests.md`
`references/review_protocols.md`
`references/schema_mapping.md`
`references/spec_audit.md`
`references/verification.md` | | [quasi-coder](../skills/quasi-coder/SKILL.md) | Expert 10x engineer skill for interpreting and implementing code from shorthand, quasi-code, and natural language descriptions. Use when collaborators provide incomplete code snippets, pseudo-code, or descriptions with potential typos or incorrect terminology. Excels at translating non-technical or semi-technical descriptions into production-quality code. | None | | [readme-blueprint-generator](../skills/readme-blueprint-generator/SKILL.md) | Intelligent README.md generation prompt that analyzes project documentation structure and creates comprehensive repository documentation. Scans .github/copilot directory files and copilot-instructions.md to extract project information, technology stack, architecture, development workflow, coding standards, and testing approaches while generating well-structured markdown documentation with proper formatting, cross-references, and developer-focused content. | None | diff --git a/skills/python-pypi-package-builder/SKILL.md b/skills/python-pypi-package-builder/SKILL.md new file mode 100644 index 00000000..e7b4a384 --- /dev/null +++ b/skills/python-pypi-package-builder/SKILL.md @@ -0,0 +1,444 @@ +--- +name: python-pypi-package-builder +description: 'End-to-end skill for building, testing, linting, versioning, and publishing a production-grade Python library to PyPI. Covers all four build backends (setuptools+setuptools_scm, hatchling, flit, poetry), PEP 440 versioning, semantic versioning, dynamic git-tag versioning, OOP/SOLID design, type hints (PEP 484/526/544/561), Trusted Publishing (OIDC), and the full PyPA packaging flow. Use for: creating Python packages, pip-installable SDKs, CLI tools, framework plugins, pyproject.toml setup, py.typed, setuptools_scm, semver, mypy, pre-commit, GitHub Actions CI/CD, or PyPI publishing.' +--- + +# Python PyPI Package Builder Skill + +A complete, battle-tested guide for building, testing, linting, versioning, typing, and +publishing a production-grade Python library to PyPI β€” from first commit to community-ready +release. + +> **AI Agent Instruction:** Read this entire file before writing a single line of code or +> creating any file. Every decision β€” layout, backend, versioning strategy, patterns, CI β€” +> has a decision rule here. Follow the decision trees in order. This skill applies to any +> Python package type (utility, SDK, CLI, plugin, data library). Do not skip sections. + +--- + +## Quick Navigation + +| Section in this file | What it covers | +|---|---| +| [1. Skill Trigger](#1-skill-trigger) | When to load this skill | +| [2. Package Type Decision](#2-package-type-decision) | Identify what you are building | +| [3. Folder Structure Decision](#3-folder-structure-decision) | src/ vs flat vs monorepo | +| [4. Build Backend Decision](#4-build-backend-decision) | setuptools / hatchling / flit / poetry | +| [5. PyPA Packaging Flow](#5-pypa-packaging-flow) | The canonical publish pipeline | +| [6. Project Structure Templates](#6-project-structure-templates) | Full layouts for every option | +| [7. Versioning Strategy](#7-versioning-strategy) | PEP 440, semver, dynamic vs static | + +| Reference file | What it covers | +|---|---| +| `references/pyproject-toml.md` | All four backend templates, `setuptools_scm`, `py.typed`, tool configs | +| `references/library-patterns.md` | OOP/SOLID, type hints, core class design, factory, protocols, CLI | +| `references/testing-quality.md` | `conftest.py`, unit/backend/async tests, ruff/mypy/pre-commit | +| `references/ci-publishing.md` | `ci.yml`, `publish.yml`, Trusted Publishing, TestPyPI, CHANGELOG, release checklist | +| `references/community-docs.md` | README, docstrings, CONTRIBUTING, SECURITY, anti-patterns, master checklist | +| `references/architecture-patterns.md` | Backend system (plugin/strategy), config layer, transport layer, CLI, backend injection | +| `references/versioning-strategy.md` | PEP 440, SemVer, pre-release, setuptools_scm deep-dive, flit static, decision engine | +| `references/release-governance.md` | Branch strategy, branch protection, OIDC, tag author validation, prevent invalid tags | +| `references/tooling-ruff.md` | Ruff-only setup (replaces black/isort), mypy config, pre-commit, asyncio_mode=auto | + +**Scaffold script:** run `python skills/python-pypi-package-builder/scripts/scaffold.py --name your-package-name` +to generate the entire directory layout, stub files, and `pyproject.toml` in one command. + +--- + +## 1. Skill Trigger + +Load this skill whenever the user wants to: + +- Create, scaffold, or publish a Python package or library to PyPI +- Build a pip-installable SDK, utility, CLI tool, or framework extension +- Set up `pyproject.toml`, linting, mypy, pre-commit, or GitHub Actions for a Python project +- Understand versioning (`setuptools_scm`, PEP 440, semver, static versioning) +- Understand PyPA specs: `py.typed`, `MANIFEST.in`, `RECORD`, classifiers +- Publish to PyPI using Trusted Publishing (OIDC) or API tokens +- Refactor an existing package to follow modern Python packaging standards +- Add type hints, protocols, ABCs, or dataclasses to a Python library +- Apply OOP/SOLID design patterns to a Python package +- Choose between build backends (setuptools, hatchling, flit, poetry) + +**Also trigger for phrases like:** "build a Python SDK", "publish my library", "set up PyPI CI", +"create a pip package", "how do I publish to PyPI", "pyproject.toml help", "PEP 561 typed", +"setuptools_scm version", "semver Python", "PEP 440", "git tag release", "Trusted Publishing". + +--- + +## 2. Package Type Decision + +Identify what the user is building **before** writing any code. Each type has distinct patterns. + +### Decision Table + +| Type | Core Pattern | Entry Point | Key Deps | Example Packages | +|---|---|---|---|---| +| **Utility library** | Module of pure functions + helpers | Import API only | Minimal | `arrow`, `humanize`, `boltons`, `more-itertools` | +| **API client / SDK** | Class with methods, auth, retry logic | Import API only | `httpx` or `requests` | `boto3`, `stripe-python`, `openai` | +| **CLI tool** | Command functions + argument parser | `[project.scripts]` or `[project.entry-points]` | `click` or `typer` | `black`, `ruff`, `httpie`, `rich` | +| **Framework plugin** | Plugin class, hook registration | `[project.entry-points."framework.plugin"]` | Framework dep | `pytest-*`, `django-*`, `flask-*` | +| **Data processing library** | Classes + functional pipeline | Import API only | Optional: `numpy`, `pandas` | `pydantic`, `marshmallow`, `cerberus` | +| **Mixed / generic** | Combination of above | Varies | Varies | Many real-world packages | + +**Decision Rule:** Ask the user if unclear. A package can combine types (e.g., SDK with a CLI +entry point) β€” use the primary type for structural decisions and add secondary type patterns on top. + +For implementation patterns of each type, see `references/library-patterns.md`. + +### Package Naming Rules + +- PyPI name: all lowercase, hyphens β€” `my-python-library` +- Python import name: underscores β€” `my_python_library` +- Check availability: https://pypi.org/search/ before starting +- Avoid shadowing popular packages (verify `pip install ` fails first) + +--- + +## 3. Folder Structure Decision + +### Decision Tree + +``` +Does the package have 5+ internal modules OR multiple contributors OR complex sub-packages? +β”œβ”€β”€ YES β†’ Use src/ layout +β”‚ Reason: prevents accidental import of uninstalled code during development; +β”‚ separates source from project root files; PyPA-recommended for large projects. +β”‚ +β”œβ”€β”€ NO β†’ Is it a single-module, focused package (e.g., one file + helpers)? +β”‚ β”œβ”€β”€ YES β†’ Use flat layout +β”‚ └── NO (medium complexity) β†’ Use flat layout, migrate to src/ if it grows +β”‚ +└── Is it multiple related packages under one namespace (e.g., myorg.http, myorg.db)? + └── YES β†’ Use namespace/monorepo layout +``` + +### Quick Rule Summary + +| Situation | Use | +|---|---| +| New project, unknown future size | `src/` layout (safest default) | +| Single-purpose, 1–4 modules | Flat layout | +| Large library, many contributors | `src/` layout | +| Multiple packages in one repo | Namespace / monorepo | +| Migrating old flat project | Keep flat; migrate to `src/` at next major version | + +--- + +## 4. Build Backend Decision + +### Decision Tree + +``` +Does the user need version derived automatically from git tags? +β”œβ”€β”€ YES β†’ Use setuptools + setuptools_scm +β”‚ (git tag v1.0.0 β†’ that IS your release workflow) +β”‚ +└── NO β†’ Does the user want an all-in-one tool (deps + build + publish)? + β”œβ”€β”€ YES β†’ Use poetry (v2+ supports standard [project] table) + β”‚ + └── NO β†’ Is the package pure Python with no C extensions? + β”œβ”€β”€ YES, minimal config preferred β†’ Use flit + β”‚ (zero config, auto-discovers version from __version__) + β”‚ + └── YES, modern & fast preferred β†’ Use hatchling + (zero-config, plugin system, no setup.py needed) + +Does the package have C/Cython/Fortran extensions? +└── YES β†’ MUST use setuptools (only backend with full native extension support) +``` + +### Backend Comparison + +| Backend | Version source | Config | C extensions | Best for | +|---|---|---|---|---| +| `setuptools` + `setuptools_scm` | git tags (automatic) | `pyproject.toml` + optional `setup.py` shim | Yes | Projects with git-tag releases; any complexity | +| `hatchling` | manual or plugin | `pyproject.toml` only | No | New pure-Python projects; fast, modern | +| `flit` | `__version__` in `__init__.py` | `pyproject.toml` only | No | Very simple, single-module packages | +| `poetry` | `pyproject.toml` field | `pyproject.toml` only | No | Teams wanting integrated dep management | + +For all four complete `pyproject.toml` templates, see `references/pyproject-toml.md`. + +--- + +## 5. PyPA Packaging Flow + +This is the canonical end-to-end flow from source code to user install. +**Every step must be understood before publishing.** + +``` +1. SOURCE TREE + Your code in version control (git) + └── pyproject.toml describes metadata + build system + +2. BUILD + python -m build + └── Produces two artifacts in dist/: + β”œβ”€β”€ *.tar.gz β†’ source distribution (sdist) + └── *.whl β†’ built distribution (wheel) β€” preferred by pip + +3. VALIDATE + twine check dist/* + └── Checks metadata, README rendering, and PyPI compatibility + +4. TEST PUBLISH (first release only) + twine upload --repository testpypi dist/* + └── Verify: pip install --index-url https://test.pypi.org/simple/ your-package + +5. PUBLISH + twine upload dist/* ← manual fallback + OR GitHub Actions publish.yml ← recommended (Trusted Publishing / OIDC) + +6. USER INSTALL + pip install your-package + pip install "your-package[extra]" +``` + +### Key PyPA Concepts + +| Concept | What it means | +|---|---| +| **sdist** | Source distribution β€” your source + metadata; used when no wheel is available | +| **wheel (.whl)** | Pre-built binary β€” pip extracts directly into site-packages; no build step | +| **PEP 517/518** | Standard build system interface via `pyproject.toml [build-system]` table | +| **PEP 621** | Standard `[project]` table in `pyproject.toml`; all modern backends support it | +| **PEP 639** | `license` key as SPDX string (e.g., `"MIT"`, `"Apache-2.0"`) β€” not `{text = "MIT"}` | +| **PEP 561** | `py.typed` empty marker file β€” tells mypy/IDEs this package ships type information | + +For complete CI workflow and publishing setup, see `references/ci-publishing.md`. + +--- + +## 6. Project Structure Templates + +### A. src/ Layout (Recommended default for new projects) + +``` +your-package/ +β”œβ”€β”€ src/ +β”‚ └── your_package/ +β”‚ β”œβ”€β”€ __init__.py # Public API: __all__, __version__ +β”‚ β”œβ”€β”€ py.typed # PEP 561 marker β€” EMPTY FILE +β”‚ β”œβ”€β”€ core.py # Primary implementation +β”‚ β”œβ”€β”€ client.py # (API client type) or remove +β”‚ β”œβ”€β”€ cli.py # (CLI type) click/typer commands, or remove +β”‚ β”œβ”€β”€ config.py # Settings / configuration dataclass +β”‚ β”œβ”€β”€ exceptions.py # Custom exception hierarchy +β”‚ β”œβ”€β”€ models.py # Data classes, Pydantic models, TypedDicts +β”‚ β”œβ”€β”€ utils.py # Internal helpers (prefix _utils if private) +β”‚ β”œβ”€β”€ types.py # Shared type aliases and TypeVars +β”‚ └── backends/ # (Plugin pattern) β€” remove if not needed +β”‚ β”œβ”€β”€ __init__.py # Protocol / ABC interface definition +β”‚ β”œβ”€β”€ memory.py # Default zero-dep implementation +β”‚ └── redis.py # Optional heavy implementation +β”œβ”€β”€ tests/ +β”‚ β”œβ”€β”€ __init__.py +β”‚ β”œβ”€β”€ conftest.py # Shared fixtures +β”‚ β”œβ”€β”€ unit/ +β”‚ β”‚ β”œβ”€β”€ __init__.py +β”‚ β”‚ β”œβ”€β”€ test_core.py +β”‚ β”‚ β”œβ”€β”€ test_config.py +β”‚ β”‚ └── test_models.py +β”‚ β”œβ”€β”€ integration/ +β”‚ β”‚ β”œβ”€β”€ __init__.py +β”‚ β”‚ └── test_backends.py +β”‚ └── e2e/ # Optional: end-to-end tests +β”‚ └── __init__.py +β”œβ”€β”€ docs/ # Optional: mkdocs or sphinx +β”œβ”€β”€ scripts/ +β”‚ └── scaffold.py +β”œβ”€β”€ .github/ +β”‚ β”œβ”€β”€ workflows/ +β”‚ β”‚ β”œβ”€β”€ ci.yml +β”‚ β”‚ └── publish.yml +β”‚ └── ISSUE_TEMPLATE/ +β”‚ β”œβ”€β”€ bug_report.md +β”‚ └── feature_request.md +β”œβ”€β”€ .pre-commit-config.yaml +β”œβ”€β”€ pyproject.toml +β”œβ”€β”€ CHANGELOG.md +β”œβ”€β”€ CONTRIBUTING.md +β”œβ”€β”€ SECURITY.md +β”œβ”€β”€ LICENSE +β”œβ”€β”€ README.md +└── .gitignore +``` + +### B. Flat Layout (Small / focused packages) + +``` +your-package/ +β”œβ”€β”€ your_package/ # ← at root, not inside src/ +β”‚ β”œβ”€β”€ __init__.py +β”‚ β”œβ”€β”€ py.typed +β”‚ └── ... (same internal structure) +β”œβ”€β”€ tests/ +└── ... (same top-level files) +``` + +### C. Namespace / Monorepo Layout (Multiple related packages) + +``` +your-org/ +β”œβ”€β”€ packages/ +β”‚ β”œβ”€β”€ your-org-core/ +β”‚ β”‚ β”œβ”€β”€ src/your_org/core/ +β”‚ β”‚ └── pyproject.toml +β”‚ β”œβ”€β”€ your-org-http/ +β”‚ β”‚ β”œβ”€β”€ src/your_org/http/ +β”‚ β”‚ └── pyproject.toml +β”‚ └── your-org-cli/ +β”‚ β”œβ”€β”€ src/your_org/cli/ +β”‚ └── pyproject.toml +β”œβ”€β”€ .github/workflows/ +└── README.md +``` + +Each sub-package has its own `pyproject.toml`. They share the `your_org` namespace via PEP 420 +implicit namespace packages (no `__init__.py` in the namespace root). + +### Internal Module Guidelines + +| File | Purpose | When to include | +|---|---|---| +| `__init__.py` | Public API surface; re-exports; `__version__` | Always | +| `py.typed` | PEP 561 typed-package marker (empty) | Always | +| `core.py` | Primary class / main logic | Always | +| `config.py` | Settings dataclass or Pydantic model | When configurable | +| `exceptions.py` | Exception hierarchy (`YourBaseError` β†’ specifics) | Always | +| `models.py` | Data models / DTOs / TypedDicts | When data-heavy | +| `utils.py` | Internal helpers (not part of public API) | As needed | +| `types.py` | Shared `TypeVar`, `TypeAlias`, `Protocol` definitions | When complex typing | +| `cli.py` | CLI entry points (click/typer) | CLI type only | +| `backends/` | Plugin/strategy pattern | When swappable implementations | +| `_compat.py` | Python version compatibility shims | When 3.9–3.13 compat needed | + +--- + +## 7. Versioning Strategy + +### PEP 440 β€” The Standard + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +Examples: + 1.0.0 Stable release + 1.0.0a1 Alpha (pre-release) + 1.0.0b2 Beta + 1.0.0rc1 Release candidate + 1.0.0.post1 Post-release (e.g., packaging fix only) + 1.0.0.dev1 Development snapshot (not for PyPI) +``` + +### Semantic Versioning (recommended) + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking API change (remove/rename public function/class/arg) +MINOR: New feature, fully backward-compatible +PATCH: Bug fix, no API change +``` + +### Dynamic versioning with setuptools_scm (recommended for git-tag workflows) + +```bash +# How it works: +git tag v1.0.0 β†’ installed version = 1.0.0 +git tag v1.1.0 β†’ installed version = 1.1.0 +(commits after tag) β†’ version = 1.1.0.post1 (suffix stripped for PyPI) + +# In code β€” NEVER hardcode when using setuptools_scm: +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Required `pyproject.toml` config: +```toml +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +g from breaking PyPI uploads +``` + +**Critical:** always set `fetch-depth: 0` in every CI checkout step. Without full git history, +`setuptools_scm` cannot find tags and the build version silently falls back to `0.0.0+dev`. + +### Static versioning (flit, hatchling manual, poetry) + +```python +# your_package/__init__.py +__version__ = "1.0.0" # Update this before every release +``` + +### Version specifier best practices for dependencies + +```toml +# In [project] dependencies: +"httpx>=0.24" # Minimum version β€” PREFERRED for libraries +"httpx>=0.24,<1.0" # Upper bound only when a known breaking change exists +"httpx==0.27.0" # Pin exactly ONLY in applications, NOT libraries + +# NEVER do this in a library β€” it breaks dependency resolution for users: +# "httpx~=0.24.0" # Too tight +# "httpx==0.27.*" # Fragile +``` + +### Version bump β†’ release flow + +```bash +# 1. Update CHANGELOG.md β€” move [Unreleased] entries to [x.y.z] - YYYY-MM-DD +# 2. Commit the changelog +git add CHANGELOG.md +git commit -m "chore: prepare release vX.Y.Z" +# 3. Tag and push β€” this triggers publish.yml automatically +git tag vX.Y.Z +git push origin main --tags +# 4. Monitor GitHub Actions β†’ verify on https://pypi.org/project/your-package/ +``` + +For complete pyproject.toml templates for all four backends, see `references/pyproject-toml.md`. + +--- + +## Where to Go Next + +After understanding decisions and structure: + +1. **Set up `pyproject.toml`** β†’ `references/pyproject-toml.md` + All four backend templates (setuptools+scm, hatchling, flit, poetry), full tool configs, + `py.typed` setup, versioning config. + +2. **Write your library code** β†’ `references/library-patterns.md` + OOP/SOLID principles, type hints (PEP 484/526/544/561), core class design, factory functions, + `__init__.py`, plugin/backend pattern, CLI entry point. + +3. **Add tests and code quality** β†’ `references/testing-quality.md` + `conftest.py`, unit/backend/async tests, parametrize, ruff/mypy/pre-commit setup. + +4. **Set up CI/CD and publish** β†’ `references/ci-publishing.md` + `ci.yml`, `publish.yml` with Trusted Publishing (OIDC, no API tokens), CHANGELOG format, + release checklist. + +5. **Polish for community/OSS** β†’ `references/community-docs.md` + README sections, docstring format, CONTRIBUTING, SECURITY, issue templates, anti-patterns + table, and master release checklist. + +6. **Design backends, config, transport, CLI** β†’ `references/architecture-patterns.md` + Backend system (plugin/strategy pattern), Settings dataclass, HTTP transport layer, + CLI with click/typer, backend injection rules. + +7. **Choose and implement a versioning strategy** β†’ `references/versioning-strategy.md` + PEP 440 canonical forms, SemVer rules, pre-release identifiers, setuptools_scm deep-dive, + flit static versioning, decision engine (DEFAULT/BEGINNER/MINIMAL). + +8. **Govern releases and secure the publish pipeline** β†’ `references/release-governance.md` + Branch strategy, branch protection rules, OIDC Trusted Publishing setup, tag author + validation in CI, tag format enforcement, full governed `publish.yml`. + +9. **Simplify tooling with Ruff** β†’ `references/tooling-ruff.md` + Ruff-only setup replacing black/isort/flake8, mypy config, pre-commit hooks, + asyncio_mode=auto (remove @pytest.mark.asyncio), migration guide. diff --git a/skills/python-pypi-package-builder/references/architecture-patterns.md b/skills/python-pypi-package-builder/references/architecture-patterns.md new file mode 100644 index 00000000..5ed08503 --- /dev/null +++ b/skills/python-pypi-package-builder/references/architecture-patterns.md @@ -0,0 +1,555 @@ +# Architecture Patterns β€” Backend System, Config, Transport, CLI + +## Table of Contents +1. [Backend System (Plugin/Strategy Pattern)](#1-backend-system-pluginstrategy-pattern) +2. [Config Layer (Settings Dataclass)](#2-config-layer-settings-dataclass) +3. [Transport Layer (HTTP Client Abstraction)](#3-transport-layer-http-client-abstraction) +4. [CLI Support](#4-cli-support) +5. [Backend Injection in Core Client](#5-backend-injection-in-core-client) +6. [Decision Rules](#6-decision-rules) + +--- + +## 1. Backend System (Plugin/Strategy Pattern) + +Structure your `backends/` sub-package with a clear base protocol, a zero-dependency default +implementation, and optional heavy implementations behind extras. + +### Directory Layout + +``` +your_package/ + backends/ + __init__.py # Exports BaseBackend + factory; holds the Protocol/ABC + base.py # Abstract base class (ABC) or Protocol definition + memory.py # Default, zero-dependency in-memory implementation + redis.py # Optional, heavier implementation (guarded by extras) +``` + +### `backends/base.py` β€” Abstract Interface + +```python +# your_package/backends/base.py +from __future__ import annotations + +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage/processing backend. + + All concrete backends must implement these methods. + Never import heavy dependencies at module level β€” guard them inside the class. + """ + + @abstractmethod + def get(self, key: str) -> str | None: + """Retrieve a value by key. Return None when the key does not exist.""" + ... + + @abstractmethod + def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value with an optional TTL (seconds).""" + ... + + @abstractmethod + def delete(self, key: str) -> None: + """Remove a key. No-op when the key does not exist.""" + ... + + def close(self) -> None: # noqa: B027 (intentionally non-abstract) + """Optional cleanup hook. Override in backends that hold connections.""" +``` + +### `backends/memory.py` β€” Default Zero-Dep Implementation + +```python +# your_package/backends/memory.py +from __future__ import annotations + +import time +from collections.abc import Iterator +from contextlib import contextmanager +from threading import Lock + +from .base import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. No external dependencies required.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = Lock() + + def get(self, key: str) -> str | None: + with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.monotonic() > expires_at: + del self._store[key] + return None + return value + + def set(self, key: str, value: str, ttl: int | None = None) -> None: + expires_at = time.monotonic() + ttl if ttl is not None else None + with self._lock: + self._store[key] = (value, expires_at) + + def delete(self, key: str) -> None: + with self._lock: + self._store.pop(key, None) +``` + +### `backends/redis.py` β€” Optional Heavy Implementation + +```python +# your_package/backends/redis.py +from __future__ import annotations + +from .base import BaseBackend + + +class RedisBackend(BaseBackend): + """Redis-backed implementation. Requires: pip install your-package[redis]""" + + def __init__(self, url: str = "redis://localhost:6379/0") -> None: + try: + import redis as _redis + except ImportError as exc: + raise ImportError( + "RedisBackend requires redis. " + "Install it with: pip install your-package[redis]" + ) from exc + self._client = _redis.from_url(url, decode_responses=True) + + def get(self, key: str) -> str | None: + return self._client.get(key) # type: ignore[return-value] + + def set(self, key: str, value: str, ttl: int | None = None) -> None: + if ttl is not None: + self._client.setex(key, ttl, value) + else: + self._client.set(key, value) + + def delete(self, key: str) -> None: + self._client.delete(key) + + def close(self) -> None: + self._client.close() +``` + +### `backends/__init__.py` β€” Public API + Factory + +```python +# your_package/backends/__init__.py +from __future__ import annotations + +from .base import BaseBackend +from .memory import MemoryBackend + +__all__ = ["BaseBackend", "MemoryBackend", "get_backend"] + + +def get_backend(backend_type: str = "memory", **kwargs: object) -> BaseBackend: + """Factory: return the requested backend instance. + + Args: + backend_type: "memory" (default) or "redis". + **kwargs: Forwarded to the backend constructor. + """ + if backend_type == "memory": + return MemoryBackend() + if backend_type == "redis": + from .redis import RedisBackend # Late import β€” redis is optional + return RedisBackend(**kwargs) # type: ignore[arg-type] + raise ValueError(f"Unknown backend type: {backend_type!r}") +``` + +--- + +## 2. Config Layer (Settings Dataclass) + +Centralise all configuration in one `config.py` module. Avoid scattering magic values and +`os.environ` calls across the codebase. + +### `config.py` + +```python +# your_package/config.py +from __future__ import annotations + +import os +from dataclasses import dataclass, field + + +@dataclass +class Settings: + """All runtime configuration for your package. + + Attributes: + api_key: Authentication credential. Never log or expose this. + timeout: HTTP request timeout in seconds. + retries: Maximum number of retry attempts on transient failures. + base_url: API base URL. Override in tests with a local server. + """ + + api_key: str + timeout: int = 30 + retries: int = 3 + base_url: str = "https://api.example.com/v1" + + def __post_init__(self) -> None: + if not self.api_key: + raise ValueError("api_key must not be empty") + if self.timeout < 1: + raise ValueError("timeout must be >= 1") + if self.retries < 0: + raise ValueError("retries must be >= 0") + + @classmethod + def from_env(cls) -> "Settings": + """Construct Settings from environment variables. + + Required env var: YOUR_PACKAGE_API_KEY + Optional env vars: YOUR_PACKAGE_TIMEOUT, YOUR_PACKAGE_RETRIES + """ + api_key = os.environ.get("YOUR_PACKAGE_API_KEY", "") + timeout = int(os.environ.get("YOUR_PACKAGE_TIMEOUT", "30")) + retries = int(os.environ.get("YOUR_PACKAGE_RETRIES", "3")) + return cls(api_key=api_key, timeout=timeout, retries=retries) +``` + +### Using Pydantic (optional, for larger projects) + +```python +# your_package/config.py β€” Pydantic v2 variant +from __future__ import annotations + +from pydantic import Field +from pydantic_settings import BaseSettings + + +class Settings(BaseSettings): + api_key: str = Field(..., min_length=1) + timeout: int = Field(30, ge=1) + retries: int = Field(3, ge=0) + base_url: str = "https://api.example.com/v1" + + model_config = {"env_prefix": "YOUR_PACKAGE_"} +``` + +--- + +## 3. Transport Layer (HTTP Client Abstraction) + +Isolate all HTTP concerns β€” headers, retries, timeouts, error parsing β€” in a dedicated +`transport/` sub-package. The core client depends on the transport abstraction, not on `httpx` +or `requests` directly. + +### Directory Layout + +``` +your_package/ + transport/ + __init__.py # Re-exports HttpTransport + http.py # Concrete httpx-based transport +``` + +### `transport/http.py` + +```python +# your_package/transport/http.py +from __future__ import annotations + +from typing import Any + +import httpx + +from ..config import Settings +from ..exceptions import YourPackageError, RateLimitError, AuthenticationError + + +class HttpTransport: + """Thin httpx wrapper that centralises auth, retries, and error mapping.""" + + def __init__(self, settings: Settings) -> None: + self._settings = settings + self._client = httpx.Client( + base_url=settings.base_url, + timeout=settings.timeout, + headers={"Authorization": f"Bearer {settings.api_key}"}, + ) + + def request( + self, + method: str, + path: str, + *, + json: dict[str, Any] | None = None, + params: dict[str, Any] | None = None, + ) -> dict[str, Any]: + """Send an HTTP request and return the parsed JSON body. + + Raises: + AuthenticationError: on 401. + RateLimitError: on 429. + YourPackageError: on all other non-2xx responses. + """ + response = self._client.request(method, path, json=json, params=params) + self._raise_for_status(response) + return response.json() + + def _raise_for_status(self, response: httpx.Response) -> None: + if response.status_code == 401: + raise AuthenticationError("Invalid or expired API key.") + if response.status_code == 429: + raise RateLimitError("Rate limit exceeded. Back off and retry.") + if response.is_error: + raise YourPackageError( + f"API error {response.status_code}: {response.text[:200]}" + ) + + def close(self) -> None: + self._client.close() + + def __enter__(self) -> "HttpTransport": + return self + + def __exit__(self, *args: object) -> None: + self.close() +``` + +### Async variant + +```python +# your_package/transport/async_http.py +from __future__ import annotations + +from typing import Any + +import httpx + +from ..config import Settings +from ..exceptions import YourPackageError, RateLimitError, AuthenticationError + + +class AsyncHttpTransport: + """Async httpx wrapper. Use with `async with AsyncHttpTransport(...) as t:`.""" + + def __init__(self, settings: Settings) -> None: + self._settings = settings + self._client = httpx.AsyncClient( + base_url=settings.base_url, + timeout=settings.timeout, + headers={"Authorization": f"Bearer {settings.api_key}"}, + ) + + async def request( + self, + method: str, + path: str, + *, + json: dict[str, Any] | None = None, + params: dict[str, Any] | None = None, + ) -> dict[str, Any]: + response = await self._client.request(method, path, json=json, params=params) + self._raise_for_status(response) + return response.json() + + def _raise_for_status(self, response: httpx.Response) -> None: + if response.status_code == 401: + raise AuthenticationError("Invalid or expired API key.") + if response.status_code == 429: + raise RateLimitError("Rate limit exceeded. Back off and retry.") + if response.is_error: + raise YourPackageError( + f"API error {response.status_code}: {response.text[:200]}" + ) + + async def aclose(self) -> None: + await self._client.aclose() + + async def __aenter__(self) -> "AsyncHttpTransport": + return self + + async def __aexit__(self, *args: object) -> None: + await self.aclose() +``` + +--- + +## 4. CLI Support + +Add a CLI entry point via `[project.scripts]` in `pyproject.toml`. + +### `pyproject.toml` entry + +```toml +[project.scripts] +your-cli = "your_package.cli:main" +``` + +After installation, the user can run `your-cli --help` directly from the terminal. + +### `cli.py` β€” Using Click + +```python +# your_package/cli.py +from __future__ import annotations + +import sys + +import click + +from .config import Settings +from .core import YourClient + + +@click.group() +@click.version_option() +def main() -> None: + """your-package CLI β€” interact with the API from the command line.""" + + +@main.command() +@click.option("--api-key", envvar="YOUR_PACKAGE_API_KEY", required=True, help="API key.") +@click.option("--timeout", default=30, show_default=True, help="Request timeout (s).") +@click.argument("query") +def search(api_key: str, timeout: int, query: str) -> None: + """Search the API and print results.""" + settings = Settings(api_key=api_key, timeout=timeout) + client = YourClient(settings=settings) + try: + results = client.search(query) + for item in results: + click.echo(item) + except Exception as exc: + click.echo(f"Error: {exc}", err=True) + sys.exit(1) +``` + +### `cli.py` β€” Using Typer (modern alternative) + +```python +# your_package/cli.py +from __future__ import annotations + +import typer + +from .config import Settings +from .core import YourClient + +app = typer.Typer(help="your-package CLI.") + + +@app.command() +def search( + query: str = typer.Argument(..., help="Search query."), + api_key: str = typer.Option(..., envvar="YOUR_PACKAGE_API_KEY"), + timeout: int = typer.Option(30, help="Request timeout (s)."), +) -> None: + """Search the API and print results.""" + settings = Settings(api_key=api_key, timeout=timeout) + client = YourClient(settings=settings) + results = client.search(query) + for item in results: + typer.echo(item) + + +def main() -> None: + app() +``` + +--- + +## 5. Backend Injection in Core Client + +**Critical:** always accept `backend` as a constructor argument. Never instantiate the backend +inside the constructor without a fallback parameter β€” that makes testing impossible. + +```python +# your_package/core.py +from __future__ import annotations + +from .backends.base import BaseBackend +from .backends.memory import MemoryBackend +from .config import Settings + + +class YourClient: + """Primary client. Accepts an injected backend for testability. + + Args: + settings: Resolved configuration. Use Settings.from_env() for production. + backend: Storage/processing backend. Defaults to MemoryBackend when None. + timeout: Deprecated β€” pass a Settings object instead. + retries: Deprecated β€” pass a Settings object instead. + """ + + def __init__( + self, + api_key: str | None = None, + *, + settings: Settings | None = None, + backend: BaseBackend | None = None, + timeout: int = 30, + retries: int = 3, + ) -> None: + if settings is None: + if api_key is None: + raise ValueError("Provide either 'api_key' or 'settings'.") + settings = Settings(api_key=api_key, timeout=timeout, retries=retries) + self._settings = settings + # CORRECT β€” default injected, not hardcoded + self.backend: BaseBackend = backend if backend is not None else MemoryBackend() + + # ... methods +``` + +### Anti-Pattern β€” Never Do This + +```python +# BAD: hardcodes the backend; impossible to swap in tests +class YourClient: + def __init__(self, api_key: str) -> None: + self.backend = MemoryBackend() # ← no injection possible + +# BAD: hardcodes the package name literal in imports +from your_package.backends.memory import MemoryBackend # only fine in your_package itself +# use relative imports inside the package: +from .backends.memory import MemoryBackend # ← correct +``` + +--- + +## 6. Decision Rules + +``` +Does the package interact with external state (cache, DB, queue)? +β”œβ”€β”€ YES β†’ Add backends/ with BaseBackend + MemoryBackend +β”‚ Add optional heavy backends behind extras_require +β”‚ +└── NO β†’ Skip backends/ entirely; keep core.py simple + +Does the package call an external HTTP API? +β”œβ”€β”€ YES β†’ Add transport/http.py; inject via Settings +β”‚ +└── NO β†’ Skip transport/ + +Does the package need a command-line interface? +β”œβ”€β”€ YES, simple (1–3 commands) β†’ Use argparse or click +β”‚ Add [project.scripts] in pyproject.toml +β”‚ +β”œβ”€β”€ YES, complex (sub-commands, plugins) β†’ Use click or typer +β”‚ +└── NO β†’ Skip cli.py + +Does runtime behaviour depend on user-supplied config? +β”œβ”€β”€ YES β†’ Add config.py with Settings dataclass +β”‚ Expose Settings.from_env() for production use +β”‚ +└── NO β†’ Accept params directly in the constructor +``` diff --git a/skills/python-pypi-package-builder/references/ci-publishing.md b/skills/python-pypi-package-builder/references/ci-publishing.md new file mode 100644 index 00000000..f96fc761 --- /dev/null +++ b/skills/python-pypi-package-builder/references/ci-publishing.md @@ -0,0 +1,315 @@ +# CI/CD, Publishing, and Changelog + +## Table of Contents +1. [Changelog format](#1-changelog-format) +2. [ci.yml β€” lint, type-check, test matrix](#2-ciyml) +3. [publish.yml β€” triggered on version tags](#3-publishyml) +4. [PyPI Trusted Publishing (no API tokens)](#4-pypi-trusted-publishing) +5. [Manual publish fallback](#5-manual-publish-fallback) +6. [Release checklist](#6-release-checklist) +7. [Verify py.typed ships in the wheel](#7-verify-pytyped-ships-in-the-wheel) +8. [Semver change-type guide](#8-semver-change-type-guide) + +--- + +## 1. Changelog Format + +Keep a `CHANGELOG.md` following [Keep a Changelog](https://keepachangelog.com/) conventions. +Every PR should update the `[Unreleased]` section. Before releasing, move those entries to a +new version section with the date. + +```markdown +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [Unreleased] + +### Added +- (in-progress features go here) + +--- + +## [1.0.0] - 2026-04-02 + +### Added +- Initial stable release +- `YourMiddleware` with gradual, strict, and combined modes +- In-memory backend (no extra deps) +- Optional Redis backend (`pip install pkg[redis]`) +- Per-route override via `Depends(RouteThrottle(...))` +- `py.typed` marker β€” PEP 561 typed package +- GitHub Actions CI: lint, mypy, test matrix, Trusted Publishing + +### Changed +### Fixed +### Removed + +--- + +## [0.1.0] - 2026-03-01 + +### Added +- Initial project scaffold + +[Unreleased]: https://github.com/you/your-package/compare/v1.0.0...HEAD +[1.0.0]: https://github.com/you/your-package/compare/v0.1.0...v1.0.0 +[0.1.0]: https://github.com/you/your-package/releases/tag/v0.1.0 +``` + +### Semver β€” what bumps what + +| Change type | Bump | Example | +|---|---|---| +| Breaking API change | MAJOR | `1.0.0 β†’ 2.0.0` | +| New feature, backward-compatible | MINOR | `1.0.0 β†’ 1.1.0` | +| Bug fix | PATCH | `1.0.0 β†’ 1.0.1` | + +--- + +## 2. `ci.yml` + +Runs on every push and pull request. Tests across all supported Python versions. + +```yaml +# .github/workflows/ci.yml +name: CI + +on: + push: + branches: [main, master] + pull_request: + branches: [main, master] + +jobs: + lint: + name: Lint, Format & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install dev dependencies + run: pip install -e ".[dev]" + - name: ruff lint + run: ruff check . + - name: ruff format check + run: ruff format --check . + - name: mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy {mod}/ + fi + + test: + name: Test (Python ${{ matrix.python-version }}) + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10", "3.11", "3.12", "3.13"] + + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED for setuptools_scm to read git tags + + - uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install dependencies + run: pip install -e ".[dev]" + + - name: Run tests with coverage + run: pytest --cov --cov-report=xml + + - name: Upload coverage + uses: codecov/codecov-action@v4 + with: + token: ${{ secrets.CODECOV_TOKEN }} + fail_ci_if_error: false + + test-redis: + name: Test Redis backend + runs-on: ubuntu-latest + services: + redis: + image: redis:7-alpine + ports: ["6379:6379"] + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install with Redis extra + run: pip install -e ".[dev,redis]" + + - name: Run Redis tests + run: pytest tests/test_redis_backend.py -v +``` + +> **Always add `fetch-depth: 0`** to every checkout step when using `setuptools_scm`. +> Without full git history, `setuptools_scm` can't find tags and the build fails with a version +> detection error. + +--- + +## 3. `publish.yml` + +Triggered automatically when you push a tag matching `v*.*.*`. Uses Trusted Publishing (OIDC) β€” +no API tokens in repository secrets. + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v*.*.*" + +jobs: + build: + name: Build distribution + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # Critical for setuptools_scm + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build tools + run: pip install build twine + + - name: Build package + run: python -m build + + - name: Check distribution + run: twine check dist/* + + - uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish: + name: Publish to PyPI + needs: build + runs-on: ubuntu-latest + environment: pypi + permissions: + id-token: write # Required for Trusted Publishing (OIDC) + + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +``` + +--- + +## 4. PyPI Trusted Publishing + +Trusted Publishing uses OpenID Connect (OIDC) so PyPI can verify that a publish came from your +specific GitHub Actions workflow β€” no long-lived API tokens required, no rotation burden. + +### One-time setup + +1. Create an account at https://pypi.org +2. Go to **Account β†’ Publishing β†’ Add a new pending publisher** +3. Fill in: + - GitHub owner (your username or org) + - Repository name + - Workflow filename: `publish.yml` + - Environment name: `pypi` +4. Create the `pypi` environment in GitHub: + **repo β†’ Settings β†’ Environments β†’ New environment β†’ name it `pypi`** + +That's it. The next time you push a `v*.*.*` tag, the workflow authenticates automatically. + +--- + +## 5. Manual Publish Fallback + +If CI isn't set up yet or you need to publish from your machine: + +```bash +pip install build twine + +# Build wheel + sdist +python -m build + +# Validate before uploading +twine check dist/* + +# Upload to PyPI +twine upload dist/* + +# OR test on TestPyPI first (recommended for first release) +twine upload --repository testpypi dist/* +pip install --index-url https://test.pypi.org/simple/ your-package +python -c "import your_package; print(your_package.__version__)" +``` + +--- + +## 6. Release Checklist + +``` +[ ] All tests pass on main/master +[ ] CHANGELOG.md updated β€” move [Unreleased] items to new version section with date +[ ] Update diff comparison links at bottom of CHANGELOG +[ ] git tag vX.Y.Z +[ ] git push origin master --tags +[ ] Monitor GitHub Actions publish.yml run +[ ] Verify on PyPI: pip install your-package==X.Y.Z +[ ] Test the installed version: + python -c "import your_package; print(your_package.__version__)" +``` + +--- + +## 7. Verify py.typed Ships in the Wheel + +After every build, confirm the typed marker is included: + +```bash +python -m build +unzip -l dist/your_package-*.whl | grep py.typed +# Must print: your_package/py.typed +# If missing, check [tool.setuptools.package-data] in pyproject.toml +``` + +If it's missing from the wheel, users won't get type information even though your code is +fully typed. This is a silent failure β€” always verify before releasing. + +--- + +## 8. Semver Change-Type Guide + +| Change | Version bump | Example | +|---|---|---| +| Breaking API change (remove/rename public symbol) | MAJOR | `1.2.3 β†’ 2.0.0` | +| New feature, fully backward-compatible | MINOR | `1.2.3 β†’ 1.3.0` | +| Bug fix, no API change | PATCH | `1.2.3 β†’ 1.2.4` | +| Pre-release | suffix | `2.0.0a1 β†’ 2.0.0rc1 β†’ 2.0.0` | +| Packaging-only fix (no code change) | post-release | `1.2.3 β†’ 1.2.3.post1` | diff --git a/skills/python-pypi-package-builder/references/community-docs.md b/skills/python-pypi-package-builder/references/community-docs.md new file mode 100644 index 00000000..ab970e1c --- /dev/null +++ b/skills/python-pypi-package-builder/references/community-docs.md @@ -0,0 +1,411 @@ +# Community Docs, PR Checklist, Anti-patterns, and Release Checklist + +## Table of Contents +1. [README.md required sections](#1-readmemd-required-sections) +2. [Docstrings β€” Google style](#2-docstrings--google-style) +3. [CONTRIBUTING.md template](#3-contributingmd) +4. [SECURITY.md template](#4-securitymd) +5. [GitHub Issue Templates](#5-github-issue-templates) +6. [PR Checklist](#6-pr-checklist) +7. [Anti-patterns to avoid](#7-anti-patterns-to-avoid) +8. [Master Release Checklist](#8-master-release-checklist) + +--- + +## 1. `README.md` Required Sections + +A good README is the single most important file for adoption. Users decide in 30 seconds whether +to use your library based on the README. + +```markdown +# your-package + +> One-line description β€” what it does and why it's useful. + +[![PyPI version](https://badge.fury.io/py/your-package.svg)](https://pypi.org/project/your-package/) +[![Python Versions](https://img.shields.io/pypi/pyversions/your-package)](https://pypi.org/project/your-package/) +[![CI](https://github.com/you/your-package/actions/workflows/ci.yml/badge.svg)](https://github.com/you/your-package/actions/workflows/ci.yml) +[![Coverage](https://codecov.io/gh/you/your-package/branch/master/graph/badge.svg)](https://codecov.io/gh/you/your-package) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) + +## Installation + +pip install your-package + +# With Redis backend: +pip install "your-package[redis]" + +## Quick Start + +(A copy-paste working example β€” no setup required to run it) + +from your_package import YourClient + +client = YourClient(api_key="sk-...") +result = client.process({"input": "value"}) +print(result) + +## Features + +- Feature 1 +- Feature 2 + +## Configuration + +| Parameter | Type | Default | Description | +|---|---|---|β€”--| +| api_key | str | required | Authentication credential | +| timeout | int | 30 | Request timeout in seconds | +| retries | int | 3 | Number of retry attempts | + +## Backends + +Brief comparison β€” in-memory vs Redis β€” and when to use each. + +## Contributing + +See [CONTRIBUTING.md](./CONTRIBUTING.md) + +## Changelog + +See [CHANGELOG.md](./CHANGELOG.md) + +## License + +MIT β€” see [LICENSE](./LICENSE) +``` + +--- + +## 2. Docstrings β€” Google Style + +Use Google-style docstrings for every public class, method, and function. IDEs display these +as tooltips, mkdocs/sphinx can auto-generate documentation from them, and they convey intent +clearly to contributors. + +```python +class YourClient: + """ + Main client for . + + Args: + api_key: Authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from your_package import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process({"input": "value"}) + """ +``` + +--- + +## 3. `CONTRIBUTING.md` + +```markdown +# Contributing to your-package + +## Development Setup + +git clone https://github.com/you/your-package +cd your-package +pip install -e ".[dev]" +pre-commit install + +## Running Tests + +pytest + +## Running Linting + +ruff check . +black . --check +mypy your_package/ + +## Submitting a PR + +1. Fork the repository +2. Create a feature branch: `git checkout -b feat/your-feature` +3. Make changes with tests +4. Ensure CI passes: `pre-commit run --all-files && pytest` +5. Update `CHANGELOG.md` under `[Unreleased]` +6. Open a PR β€” use the PR template + +## Commit Message Format (Conventional Commits) + +- `feat: add Redis backend` +- `fix: correct retry behavior on timeout` +- `docs: update README quick start` +- `chore: bump ruff to 0.5` +- `test: add edge cases for memory backend` + +## Reporting Bugs + +Use the GitHub issue template. Include Python version, package version, +and a minimal reproducible example. +``` + +--- + +## 4. `SECURITY.md` + +```markdown +# Security Policy + +## Supported Versions + +| Version | Supported | +|---|---| +| 1.x.x | Yes | +| < 1.0 | No | + +## Reporting a Vulnerability + +Do NOT open a public GitHub issue for security vulnerabilities. + +Report via: GitHub private security reporting (preferred) +or email: security@yourdomain.com + +Include: +- Description of the vulnerability +- Steps to reproduce +- Potential impact +- Suggested fix (if any) + +We aim to acknowledge within 48 hours and resolve within 14 days. +``` + +--- + +## 5. GitHub Issue Templates + +### `.github/ISSUE_TEMPLATE/bug_report.md` + +```markdown +--- +name: Bug Report +about: Report a reproducible bug +labels: bug +--- + +**Python version:** +**Package version:** + +**Describe the bug:** + +**Minimal reproducible example:** +```python +# paste code here +``` + +**Expected behavior:** + +**Actual behavior:** +``` + +### `.github/ISSUE_TEMPLATE/feature_request.md` + +```markdown +--- +name: Feature Request +about: Suggest a new feature or enhancement +labels: enhancement +--- + +**Problem this would solve:** + +**Proposed solution:** + +**Alternatives considered:** +``` + +--- + +## 6. PR Checklist + +All items must be checked before requesting review. CI must be fully green. + +### Code Quality Gates +``` +[ ] ruff check . β€” zero errors +[ ] black . --check β€” zero formatting issues +[ ] isort . --check-only β€” imports sorted correctly +[ ] mypy your_package/ β€” zero type errors +[ ] pytest β€” all tests pass +[ ] Coverage >= 80% (enforced by fail_under in pyproject.toml) +[ ] All GitHub Actions workflows green +``` + +### Structure +``` +[ ] pyproject.toml: name, dynamic/version, description, requires-python, license, authors, + keywords (10+), classifiers, dependencies, all [project.urls] filled in +[ ] dynamic = ["version"] if using setuptools_scm +[ ] [tool.setuptools_scm] with local_scheme = "no-local-version" +[ ] setup.py shim present (if using setuptools_scm) +[ ] py.typed marker file exists in the package directory (empty file) +[ ] py.typed listed in [tool.setuptools.package-data] +[ ] "Typing :: Typed" classifier in pyproject.toml +[ ] __init__.py has __all__ listing all public symbols +[ ] __version__ via importlib.metadata (not hardcoded string) +``` + +### Testing +``` +[ ] conftest.py has shared fixtures for client and backend +[ ] Core happy path tested +[ ] Error conditions and edge cases tested +[ ] Each backend tested independently in isolation +[ ] Redis backend tested in separate CI job with redis service (if applicable) +[ ] asyncio_mode = "auto" in pyproject.toml (for async tests) +[ ] fetch-depth: 0 in all CI checkout steps +``` + +### Optional Backend (if applicable) +``` +[ ] BaseBackend abstract class defines the interface +[ ] MemoryBackend works with zero extra deps +[ ] RedisBackend raises ImportError with clear pip install hint if redis not installed +[ ] Both backends unit-tested independently +[ ] redis extra declared in [project.optional-dependencies] +[ ] README shows both install paths (base and [redis]) +``` + +### Changelog & Docs +``` +[ ] CHANGELOG.md updated under [Unreleased] +[ ] README has: description, install, quick start, config table, badges, license +[ ] All public symbols have Google-style docstrings +[ ] CONTRIBUTING.md: dev setup, test/lint commands, PR instructions +[ ] SECURITY.md: supported versions, reporting process +[ ] .github/ISSUE_TEMPLATE/bug_report.md +[ ] .github/ISSUE_TEMPLATE/feature_request.md +``` + +### CI/CD +``` +[ ] ci.yml: lint + mypy + test matrix (all supported Python versions) +[ ] ci.yml: separate job for Redis backend with redis service +[ ] publish.yml: triggered on v*.*.* tags, uses Trusted Publishing (OIDC) +[ ] fetch-depth: 0 in all workflow checkout steps +[ ] pypi environment created in GitHub repo Settings β†’ Environments +[ ] No API tokens in repository secrets +``` + +--- + +## 7. Anti-patterns to Avoid + +| Anti-pattern | Why it's bad | Correct approach | +|---|---|---| +| `__version__ = "1.0.0"` hardcoded with setuptools_scm | Goes stale after first git tag | Use `importlib.metadata.version()` | +| Missing `fetch-depth: 0` in CI checkout | setuptools_scm can't find tags β†’ version = `0.0.0+dev` | Add `fetch-depth: 0` to **every** checkout step | +| `local_scheme` not set | `+g` suffix breaks PyPI uploads (local versions rejected) | `local_scheme = "no-local-version"` | +| Missing `py.typed` file | IDEs and mypy don't see package as typed | Create empty `py.typed` in package root | +| `py.typed` not in `package-data` | File missing from installed wheel β€” useless | Add to `[tool.setuptools.package-data]` | +| Importing optional dep at module top | `ImportError` on `import your_package` for all users | Lazy import inside the function/class that needs it | +| Duplicating metadata in `setup.py` | Conflicts with `pyproject.toml`; drifts | Keep `setup.py` as 3-line shim only | +| No `fail_under` in coverage config | Coverage regressions go unnoticed | Set `fail_under = 80` | +| No mypy in CI | Type errors silently accumulate | Add mypy step to `ci.yml` | +| API tokens in GitHub Secrets for PyPI | Security risk, rotation burden | Use Trusted Publishing (OIDC) | +| Committing directly to `main`/`master` | Bypasses CI checks | Enforce via `no-commit-to-branch` pre-commit hook | +| Missing `[Unreleased]` section in CHANGELOG | Changes pile up and get forgotten at release time | Keep `[Unreleased]` updated every PR | +| Pinning exact dep versions in a library | Breaks dependency resolution for users | Use `>=` lower bounds only; avoid `==` | +| No `__all__` in `__init__.py` | Users can accidentally import internal helpers | Declare `__all__` with every public symbol | +| `from your_package import *` in tests | Tests pass even when imports are broken | Always use explicit imports | +| No `SECURITY.md` | No path for responsible vulnerability disclosure | Add file with response timeline | +| `Any` everywhere in type hints | Defeats mypy entirely | Use `object` for truly arbitrary values | +| `Union` return types | Forces every caller to write `isinstance()` checks | Return concrete types; use overloads | +| `setup.cfg` + `pyproject.toml` both active | Conflicts and confusing for contributors | Migrate everything to `pyproject.toml` | +| Releasing on untagged commits | Version number is meaningless | Always tag before release | +| Not testing on all supported Python versions | Breakage discovered by users, not you | Matrix test in CI | +| `license = {text = "MIT"}` (old form) | Deprecated; PEP 639 uses SPDX strings | `license = "MIT"` | +| No issue templates | Bug reports are inconsistent | Add `bug_report.md` + `feature_request.md` | + +--- + +## 8. Master Release Checklist + +Run through every item before pushing a release tag. CI must be fully green. + +### Code Quality +``` +[ ] ruff check . β€” zero errors +[ ] ruff format . --check β€” zero formatting issues +[ ] mypy src/your_package/ β€” zero type errors +[ ] pytest β€” all tests pass +[ ] Coverage >= 80% (fail_under enforced in pyproject.toml) +[ ] All GitHub Actions CI jobs green (lint + test matrix) +``` + +### Project Structure +``` +[ ] pyproject.toml β€” name, description, requires-python, license (SPDX string), authors, + keywords (10+), classifiers (Python versions + Typing :: Typed), urls (all 5 fields) +[ ] dynamic = ["version"] set (if using setuptools_scm or hatch-vcs) +[ ] [tool.setuptools_scm] with local_scheme = "no-local-version" +[ ] setup.py shim present (if using setuptools_scm) +[ ] py.typed marker file exists (empty file in package root) +[ ] py.typed listed in [tool.setuptools.package-data] +[ ] "Typing :: Typed" classifier in pyproject.toml +[ ] __init__.py has __all__ listing all public symbols +[ ] __version__ reads from importlib.metadata (not hardcoded) +``` + +### Testing +``` +[ ] conftest.py has shared fixtures for client and backend +[ ] Core happy path tested +[ ] Error conditions and edge cases tested +[ ] Each backend tested independently in isolation +[ ] asyncio_mode = "auto" in pyproject.toml (for async tests) +[ ] fetch-depth: 0 in all CI checkout steps +``` + +### CHANGELOG and Docs +``` +[ ] CHANGELOG.md: [Unreleased] entries moved to [x.y.z] - YYYY-MM-DD +[ ] README has: description, install commands, quick start, config table, badges +[ ] All public symbols have Google-style docstrings +[ ] CONTRIBUTING.md: dev setup, test/lint commands, PR instructions +[ ] SECURITY.md: supported versions, reporting process with timeline +``` + +### Versioning +``` +[ ] All CI checks pass on the commit you plan to tag +[ ] CHANGELOG.md updated and committed +[ ] Git tag follows format v1.2.3 (semver, v prefix) +[ ] No stale local_scheme suffixes will appear in the built wheel name +``` + +### CI/CD +``` +[ ] ci.yml: lint + mypy + test matrix (all supported Python versions) +[ ] publish.yml: triggered on v*.*.* tags, uses Trusted Publishing (OIDC) +[ ] pypi environment created in GitHub repo Settings β†’ Environments +[ ] No API tokens stored in repository secrets +``` + +### The Release Command Sequence +```bash +# 1. Run full local validation +ruff check . ; ruff format . --check ; mypy src/your_package/ ; pytest + +# 2. Update CHANGELOG.md β€” move [Unreleased] to [x.y.z] +# 3. Commit the changelog +git add CHANGELOG.md +git commit -m "chore: prepare release vX.Y.Z" + +# 4. Tag and push β€” this triggers publish.yml automatically +git tag vX.Y.Z +git push origin main --tags + +# 5. Monitor: https://github.com///actions +# 6. Verify: https://pypi.org/project/your-package/ +``` diff --git a/skills/python-pypi-package-builder/references/library-patterns.md b/skills/python-pypi-package-builder/references/library-patterns.md new file mode 100644 index 00000000..d1ec780c --- /dev/null +++ b/skills/python-pypi-package-builder/references/library-patterns.md @@ -0,0 +1,606 @@ +# Library Core Patterns, OOP/SOLID, and Type Hints + +## Table of Contents +1. [OOP & SOLID Principles](#1-oop--solid-principles) +2. [Type Hints Best Practices](#2-type-hints-best-practices) +3. [Core Class Design](#3-core-class-design) +4. [Factory / Builder Pattern](#4-factory--builder-pattern) +5. [Configuration Pattern](#5-configuration-pattern) +6. [`__init__.py` β€” explicit public API](#6-__init__py--explicit-public-api) +7. [Optional Backends (Plugin Pattern)](#7-optional-backends-plugin-pattern) + +--- + +## 1. OOP & SOLID Principles + +Apply these principles to produce maintainable, testable, extensible packages. +**Do not over-engineer** β€” apply the principle that solves a real problem, not all of them +at once. + +### S β€” Single Responsibility Principle + +Each class/module should have **one reason to change**. + +```python +# BAD: one class handles data, validation, AND persistence +class UserManager: + def validate(self, user): ... + def save_to_db(self, user): ... + def send_email(self, user): ... + +# GOOD: split responsibilities +class UserValidator: + def validate(self, user: User) -> None: ... + +class UserRepository: + def save(self, user: User) -> None: ... + +class UserNotifier: + def notify(self, user: User) -> None: ... +``` + +### O β€” Open/Closed Principle + +Open for extension, closed for modification. Use **protocols or ABCs** as extension points. + +```python +from abc import ABC, abstractmethod + +class StorageBackend(ABC): + """Define the interface once; never modify it for new implementations.""" + @abstractmethod + def get(self, key: str) -> str | None: ... + @abstractmethod + def set(self, key: str, value: str) -> None: ... + +class MemoryBackend(StorageBackend): # Extend by subclassing + ... + +class RedisBackend(StorageBackend): # Add new impl without touching StorageBackend + ... +``` + +### L β€” Liskov Substitution Principle + +Subclasses must be substitutable for their base. Never narrow a contract in a subclass. + +```python +class BaseProcessor: + def process(self, data: dict) -> dict: ... + +# BAD: raises TypeError for valid dicts β€” breaks substitutability +class StrictProcessor(BaseProcessor): + def process(self, data: dict) -> dict: + if not data: + raise TypeError("Must have data") # Base never raised this + +# GOOD: accept what base accepts, fulfill the same contract +class StrictProcessor(BaseProcessor): + def process(self, data: dict) -> dict: + if not data: + return {} # Graceful β€” same return type, no new exceptions +``` + +### I β€” Interface Segregation Principle + +Prefer **small, focused protocols** over large monolithic ABCs. + +```python +# BAD: forces all implementers to handle read+write+delete+list +class BigStorage(ABC): + @abstractmethod + def read(self): ... + @abstractmethod + def write(self): ... + @abstractmethod + def delete(self): ... + @abstractmethod + def list_all(self): ... # Not every backend needs this + +# GOOD: separate protocols β€” clients depend only on what they need +from typing import Protocol + +class Readable(Protocol): + def read(self, key: str) -> str | None: ... + +class Writable(Protocol): + def write(self, key: str, value: str) -> None: ... + +class Deletable(Protocol): + def delete(self, key: str) -> None: ... +``` + +### D β€” Dependency Inversion Principle + +High-level modules depend on **abstractions** (protocols/ABCs), not concrete implementations. +Pass dependencies in via `__init__` (constructor injection). + +```python +# BAD: high-level class creates its own dependency +class ApiClient: + def __init__(self) -> None: + self._cache = RedisCache() # Tightly coupled to Redis + +# GOOD: depend on the abstraction; inject the concrete at call site +class ApiClient: + def __init__(self, cache: CacheBackend) -> None: # CacheBackend is a Protocol + self._cache = cache + +# User code (or tests): +client = ApiClient(cache=RedisCache()) # Real +client = ApiClient(cache=MemoryCache()) # Test +``` + +### Composition Over Inheritance + +Prefer delegating to contained objects over deep inheritance chains. + +```python +# Prefer this (composition): +class YourClient: + def __init__(self, backend: StorageBackend, http: HttpTransport) -> None: + self._backend = backend + self._http = http + +# Avoid this (deep inheritance): +class YourClient(BaseClient, CacheMixin, RetryMixin, LoggingMixin): + ... # Fragile, hard to test, MRO confusion +``` + +### Exception Hierarchy + +Always define a base exception for your package; layer specifics below it. + +```python +# your_package/exceptions.py +class YourPackageError(Exception): + """Base exception β€” catch this to catch any package error.""" + +class ConfigurationError(YourPackageError): + """Raised when package is misconfigured.""" + +class AuthenticationError(YourPackageError): + """Raised on auth failure.""" + +class RateLimitError(YourPackageError): + """Raised when rate limit is exceeded.""" + def __init__(self, retry_after: int) -> None: + self.retry_after = retry_after + super().__init__(f"Rate limited. Retry after {retry_after}s.") +``` + +--- + +## 2. Type Hints Best Practices + +Follow PEP 484 (type hints), PEP 526 (variable annotations), PEP 544 (protocols), +PEP 561 (typed packages). These are not optional for a quality library. + +```python +from __future__ import annotations # Enables PEP 563 deferred evaluation β€” always add this + +# For ARGUMENTS: prefer abstract / protocol types (more flexible for callers) +from collections.abc import Iterable, Mapping, Sequence, Callable + +def process_items(items: Iterable[str]) -> list[int]: ... # βœ“ Accepts any iterable +def process_items(items: list[str]) -> list[int]: ... # βœ— Too restrictive + +# For RETURN TYPES: prefer concrete types (callers know exactly what they get) +def get_names() -> list[str]: ... # βœ“ Concrete +def get_names() -> Iterable[str]: ... # βœ— Caller can't index it + +# Use X | Y syntax (Python 3.10+), not Union[X, Y] or Optional[X] +def find(key: str) -> str | None: ... # βœ“ Modern +def find(key: str) -> Optional[str]: ... # βœ— Old style + +# None should be LAST in unions +def get(key: str) -> str | int | None: ... # βœ“ + +# Avoid Any β€” it disables type checking entirely +def process(data: Any) -> Any: ... # βœ— Loses all safety +def process(data: dict[str, object]) -> dict[str, object]: # βœ“ + +# Use object instead of Any when a param accepts literally anything +def log(value: object) -> None: ... # βœ“ + +# Avoid Union return types β€” they require isinstance() checks at every call site +def get_value() -> str | int: ... # βœ— Forces callers to branch +``` + +### Protocols vs ABCs + +```python +from typing import Protocol, runtime_checkable +from abc import ABC, abstractmethod + +# Use Protocol when you don't control the implementer classes (duck typing) +@runtime_checkable # Makes isinstance() checks work at runtime +class Serializable(Protocol): + def to_dict(self) -> dict[str, object]: ... + +# Use ABC when you control the class hierarchy and want default implementations +class BaseBackend(ABC): + @abstractmethod + async def get(self, key: str) -> str | None: ... + + def get_or_default(self, key: str, default: str) -> str: + result = self.get(key) + return result if result is not None else default +``` + +### TypeVar and Generics + +```python +from typing import TypeVar, Generic + +T = TypeVar("T") +T_co = TypeVar("T_co", covariant=True) # For read-only containers + +class Repository(Generic[T]): + """Type-safe generic repository.""" + def __init__(self, model_class: type[T]) -> None: + self._store: list[T] = [] + + def add(self, item: T) -> None: + self._store.append(item) + + def get_all(self) -> list[T]: + return list(self._store) +``` + +### dataclasses for data containers + +```python +from dataclasses import dataclass, field + +@dataclass(frozen=True) # frozen=True β†’ immutable, hashable (good for configs/keys) +class Config: + api_key: str + timeout: int = 30 + headers: dict[str, str] = field(default_factory=dict) + + def __post_init__(self) -> None: + if not self.api_key: + raise ValueError("api_key must not be empty") +``` + +### TYPE_CHECKING guard (avoid circular imports) + +```python +from __future__ import annotations +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from your_package.models import HeavyModel # Only imported during type checking + +def process(model: "HeavyModel") -> None: + ... +``` + +### Overload for multiple signatures + +```python +from typing import overload + +@overload +def get(key: str, default: None = ...) -> str | None: ... +@overload +def get(key: str, default: str) -> str: ... +def get(key: str, default: str | None = None) -> str | None: + ... # Single implementation handles both +``` + +--- + +## 3. Core Class Design + +The main class of your library should have a clear, minimal `__init__`, sensible defaults for all +parameters, and raise `TypeError` / `ValueError` early for invalid inputs. This prevents confusing +errors at call time rather than at construction. + +```python +# your_package/core.py +from __future__ import annotations + +from your_package.exceptions import YourPackageError + + +class YourClient: + """ + Main entry point for . + + Args: + api_key: Required authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from your_package import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process(data) + """ + + def __init__( + self, + api_key: str, + timeout: int = 30, + retries: int = 3, + ) -> None: + if not api_key: + raise ValueError("api_key must not be empty") + if timeout <= 0: + raise ValueError("timeout must be positive") + self._api_key = api_key + self.timeout = timeout + self.retries = retries + + def process(self, data: dict) -> dict: + """ + Process data and return results. + + Args: + data: Input dictionary to process. + + Returns: + Processed result as a dictionary. + + Raises: + YourPackageError: If processing fails. + """ + ... +``` + +### Design rules + +- Accept all config in `__init__`, not scattered across method calls. +- Validate at construction time β€” fail fast with a clear message. +- Keep `__init__` signatures stable. Adding new **keyword-only** args with defaults is backwards + compatible. Removing or reordering positional args is a breaking change. + +--- + +## 4. Factory / Builder Pattern + +Use a factory function when users need to create pre-configured instances. This avoids cluttering +`__init__` with a dozen keyword arguments and keeps the common case simple. + +```python +# your_package/factory.py +from __future__ import annotations + +from your_package.core import YourClient +from your_package.backends.memory import MemoryBackend + + +def create_client( + api_key: str, + *, + timeout: int = 30, + retries: int = 3, + backend: str = "memory", + backend_url: str | None = None, +) -> YourClient: + """ + Factory that returns a configured YourClient. + + Args: + api_key: Required API key. + timeout: Request timeout in seconds. + retries: Number of retry attempts. + backend: Storage backend type. One of 'memory' or 'redis'. + backend_url: Connection URL for the chosen backend. + + Example: + >>> client = create_client(api_key="sk-...", backend="redis", backend_url="redis://localhost") + """ + if backend == "redis": + from your_package.backends.redis import RedisBackend + _backend = RedisBackend(url=backend_url or "redis://localhost:6379") + else: + _backend = MemoryBackend() + + return YourClient(api_key=api_key, timeout=timeout, retries=retries, backend=_backend) +``` + +**Why a factory, not a class method?** Both work. A standalone factory function is easier to +mock in tests and avoids coupling the factory logic into the class itself. + +--- + +## 5. Configuration Pattern + +Use a dataclass (or Pydantic `BaseModel`) to hold configuration. This gives you free validation, +helpful error messages, and a single place to document every option. + +```python +# your_package/config.py +from __future__ import annotations +from dataclasses import dataclass, field + + +@dataclass +class YourSettings: + """ + Configuration for YourClient. + + Attributes: + timeout: HTTP timeout in seconds. + retries: Number of retry attempts on transient errors. + base_url: Base API URL. + """ + timeout: int = 30 + retries: int = 3 + base_url: str = "https://api.example.com" + extra_headers: dict[str, str] = field(default_factory=dict) + + def __post_init__(self) -> None: + if self.timeout <= 0: + raise ValueError("timeout must be positive") + if self.retries < 0: + raise ValueError("retries must be non-negative") +``` + +If you need environment variable loading, use `pydantic-settings` as an **optional** dependency β€” +declare it in `[project.optional-dependencies]`, not as a required dep. + +--- + +## 6. `__init__.py` β€” Explicit Public API + +A well-defined `__all__` is not just style β€” it tells users (and IDEs) exactly what's part of your +public API, and prevents accidental imports of internal helpers as part of your contract. + +```python +# your_package/__init__.py +"""your-package: .""" + +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" + +from your_package.core import YourClient +from your_package.config import YourSettings +from your_package.exceptions import YourPackageError + +__all__ = [ + "YourClient", + "YourSettings", + "YourPackageError", + "__version__", +] +``` + +Rules: +- Only export what users are supposed to use. Internal helpers go in `_utils.py` or submodules. +- Keep imports at the top level of `__init__.py` shallow β€” avoid importing heavy optional deps + (like `redis`) at module level. Import them lazily inside the class or function that needs them. +- `__version__` is always part of the public API β€” it enables `your_package.__version__` for + debugging. + +--- + +## 7. Optional Backends (Plugin Pattern) + +This pattern lets your package work out-of-the-box (no extra deps) with an in-memory backend, +while letting advanced users plug in Redis, a database, or any custom storage. + +### 5.1 Abstract base class β€” defines the interface + +```python +# your_package/backends/__init__.py +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage backend interface. + + Implement this to add a custom backend (database, cache, etc.). + """ + + @abstractmethod + async def get(self, key: str) -> str | None: + """Retrieve a value by key. Returns None if not found.""" + ... + + @abstractmethod + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value. Optional TTL in seconds.""" + ... + + @abstractmethod + async def delete(self, key: str) -> None: + """Delete a key.""" + ... +``` + +### 5.2 Memory backend β€” zero extra deps + +```python +# your_package/backends/memory.py +from __future__ import annotations + +import asyncio +import time +from your_package.backends import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. Works out of the box β€” no extra dependencies.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = asyncio.Lock() + + async def get(self, key: str) -> str | None: + async with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.time() > expires_at: + del self._store[key] + return None + return value + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + async with self._lock: + expires_at = time.time() + ttl if ttl is not None else None + self._store[key] = (value, expires_at) + + async def delete(self, key: str) -> None: + async with self._lock: + self._store.pop(key, None) +``` + +### 5.3 Redis backend β€” raises clear ImportError if not installed + +The key design: import `redis` lazily inside `__init__`, not at module level. This way, +`import your_package` never fails even if `redis` isn't installed. + +```python +# your_package/backends/redis.py +from __future__ import annotations +from your_package.backends import BaseBackend + +try: + import redis.asyncio as aioredis +except ImportError as exc: + raise ImportError( + "Redis backend requires the redis extra:\n" + " pip install your-package[redis]" + ) from exc + + +class RedisBackend(BaseBackend): + """Redis-backed storage for distributed/multi-process deployments.""" + + def __init__(self, url: str = "redis://localhost:6379") -> None: + self._client = aioredis.from_url(url, decode_responses=True) + + async def get(self, key: str) -> str | None: + return await self._client.get(key) + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + await self._client.set(key, value, ex=ttl) + + async def delete(self, key: str) -> None: + await self._client.delete(key) +``` + +### 5.4 How users choose a backend + +```python +# Default: in-memory, no extra deps needed +from your_package import YourClient +client = YourClient(api_key="sk-...") + +# Redis: pip install your-package[redis] +from your_package.backends.redis import RedisBackend +client = YourClient(api_key="sk-...", backend=RedisBackend(url="redis://localhost:6379")) +``` diff --git a/skills/python-pypi-package-builder/references/pyproject-toml.md b/skills/python-pypi-package-builder/references/pyproject-toml.md new file mode 100644 index 00000000..eccfac96 --- /dev/null +++ b/skills/python-pypi-package-builder/references/pyproject-toml.md @@ -0,0 +1,470 @@ +# pyproject.toml, Backends, Versioning, and Typed Package + +## Table of Contents +1. [Complete pyproject.toml β€” setuptools + setuptools_scm](#1-complete-pyprojecttoml) +2. [hatchling (modern, zero-config)](#2-hatchling-modern-zero-config) +3. [flit (minimal, version from `__version__`)](#3-flit-minimal-version-from-__version__) +4. [poetry (integrated dep manager)](#4-poetry-integrated-dep-manager) +5. [Versioning Strategy β€” PEP 440, semver, dep specifiers](#5-versioning-strategy) +6. [setuptools_scm β€” dynamic version from git tags](#6-dynamic-versioning-with-setuptools_scm) +7. [setup.py shim for legacy editable installs](#7-setuppy-shim) +8. [PEP 561 typed package (py.typed)](#8-typed-package-pep-561) + +--- + +## 1. Complete pyproject.toml + +### setuptools + setuptools_scm (recommended for git-tag versioning) + +```toml +[build-system] +requires = ["setuptools>=68", "wheel", "setuptools_scm"] +build-backend = "setuptools.build_meta" + +[project] +name = "your-package" +dynamic = ["version"] # Version comes from git tags via setuptools_scm +description = " β€” , " +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" # PEP 639 SPDX expression (string, not {text = "MIT"}) +license-files = ["LICENSE"] +authors = [ + {name = "Your Name", email = "you@example.com"}, +] +maintainers = [ + {name = "Your Name", email = "you@example.com"}, +] +keywords = [ + "python", + # Add 10-15 specific keywords that describe your library β€” they affect PyPI discoverability +] +classifiers = [ + "Development Status :: 3 - Alpha", # Change to 5 at stable release + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Topic :: Software Development :: Libraries :: Python Modules", + "Typing :: Typed", # Add this when shipping py.typed +] +dependencies = [ + # List your runtime dependencies here. Keep them minimal. + # Example: "httpx>=0.24", "pydantic>=2.0" + # Leave empty if your library has no required runtime deps. +] + +[project.optional-dependencies] +redis = [ + "redis>=4.2", # Optional heavy backend +] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" +Documentation = "https://github.com/yourusername/your-package#readme" +Repository = "https://github.com/yourusername/your-package" +"Bug Tracker" = "https://github.com/yourusername/your-package/issues" +Changelog = "https://github.com/yourusername/your-package/blob/master/CHANGELOG.md" + +# --- Setuptools configuration --- +[tool.setuptools.packages.find] +include = ["your_package*"] # flat layout +# For src/ layout, use: +# where = ["src"] + +[tool.setuptools.package-data] +your_package = ["py.typed"] # Ship the py.typed marker in the wheel + +# --- setuptools_scm: version from git tags --- +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +local suffix breaking PyPI uploads + +# --- Ruff (linting) --- +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "N", "UP", "B", "SIM", "C4", "PTH", "RUF"] +ignore = ["E501"] # Line length enforced by formatter + +[tool.ruff.lint.per-file-ignores] +"tests/*" = ["S101", "ANN"] # Allow assert and missing annotations in tests +"scripts/*" = ["T201"] # Allow print in scripts + +[tool.ruff.format] +quote-style = "double" + +# --- Black (formatting) --- +[tool.black] +line-length = 100 +target-version = ["py310", "py311", "py312", "py313"] + +# --- isort (import sorting) --- +[tool.isort] +profile = "black" +line_length = 100 + +# --- mypy (static type checking) --- +[tool.mypy] +python_version = "3.10" +warn_return_any = true +warn_unused_configs = true +warn_unused_ignores = true +disallow_untyped_defs = true +disallow_any_generics = true +ignore_missing_imports = true +strict = false # Set true for maximum strictness + +[[tool.mypy.overrides]] +module = "tests.*" +disallow_untyped_defs = false # Relaxed in tests + +# --- pytest --- +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"] +pythonpath = ["."] # For flat layout; remove for src/ +python_files = "test_*.py" +python_classes = "Test*" +python_functions = "test_*" +addopts = "-v --tb=short --cov=your_package --cov-report=term-missing" + +# --- Coverage --- +[tool.coverage.run] +source = ["your_package"] +omit = ["tests/*"] + +[tool.coverage.report] +fail_under = 80 +show_missing = true +exclude_lines = [ + "pragma: no cover", + "def __repr__", + "raise NotImplementedError", + "if TYPE_CHECKING:", + "@abstractmethod", +] +``` + +--- + +## 2. hatchling (Modern, Zero-Config) + +Best for new pure-Python projects that don't need C extensions. No `setup.py` needed. Use +`hatch-vcs` for git-tag versioning, or omit it for manual version bumps. + +```toml +[build-system] +requires = ["hatchling", "hatch-vcs"] # hatch-vcs for git-tag versioning +build-backend = "hatchling.build" + +[project] +name = "your-package" +dynamic = ["version"] # Remove and add version = "1.0.0" for manual versioning +description = "One-line description" +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +license-files = ["LICENSE"] +authors = [{name = "Your Name", email = "you@example.com"}] +keywords = ["python"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] + +[project.optional-dependencies] +dev = ["pytest>=8.0", "pytest-cov>=5.0", "ruff>=0.6", "mypy>=1.10"] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" +Changelog = "https://github.com/yourusername/your-package/blob/master/CHANGELOG.md" + +# --- Hatchling build config --- +[tool.hatch.build.targets.wheel] +packages = ["src/your_package"] # src/ layout +# packages = ["your_package"] # ← flat layout + +[tool.hatch.version] +source = "vcs" # git-tag versioning via hatch-vcs + +[tool.hatch.version.raw-options] +local_scheme = "no-local-version" + +# ruff, mypy, pytest, coverage sections β€” same as setuptools template above +``` + +--- + +## 3. flit (Minimal, Version from `__version__`) + +Best for very simple, single-module packages. Zero config. Version is read directly from +`your_package/__init__.py`. Always requires a **static string** for `__version__`. + +```toml +[build-system] +requires = ["flit_core>=3.9"] +build-backend = "flit_core.buildapi" + +[project] +name = "your-package" +dynamic = ["version", "description"] # Read from __init__.py __version__ and docstring +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +authors = [{name = "Your Name", email = "you@example.com"}] +classifiers = [ + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] + +[project.urls] +Homepage = "https://github.com/yourusername/your-package" + +# flit reads __version__ from your_package/__init__.py automatically. +# Ensure __init__.py has: __version__ = "1.0.0" (static string β€” flit does NOT support +# importlib.metadata for dynamic version discovery) +``` + +--- + +## 4. poetry (Integrated Dependency + Build Manager) + +Best for teams that want a single tool to manage deps, build, and publish. Poetry v2+ +supports the standard `[project]` table. + +```toml +[build-system] +requires = ["poetry-core>=2.0"] +build-backend = "poetry.core.masonry.api" + +[project] +name = "your-package" +version = "1.0.0" +description = "One-line description" +readme = "README.md" +requires-python = ">=3.10" +license = "MIT" +authors = [{name = "Your Name", email = "you@example.com"}] +classifiers = [ + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [] # poetry v2+ uses standard [project] table + +[project.optional-dependencies] +dev = ["pytest>=8.0", "ruff>=0.6", "mypy>=1.10"] + +# Optional: use [tool.poetry] only for poetry-specific features +[tool.poetry.group.dev.dependencies] +# Poetry-specific group syntax (alternative to [project.optional-dependencies]) +pytest = ">=8.0" +``` + +--- + +## 5. Versioning Strategy + +### PEP 440 β€” The Standard + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +Examples: + 1.0.0 Stable release + 1.0.0a1 Alpha (pre-release) + 1.0.0b2 Beta + 1.0.0rc1 Release candidate + 1.0.0.post1 Post-release (e.g., packaging fix only β€” no code change) + 1.0.0.dev1 Development snapshot (NOT for PyPI) +``` + +### Semantic Versioning (SemVer) β€” use this for every library + +``` +MAJOR.MINOR.PATCH + +MAJOR: Breaking API change (remove/rename public function/class/arg) +MINOR: New feature, fully backward-compatible +PATCH: Bug fix, no API change +``` + +| Change | What bumps | Example | +|---|---|---| +| Remove / rename a public function | MAJOR | `1.2.3 β†’ 2.0.0` | +| Add new public function | MINOR | `1.2.3 β†’ 1.3.0` | +| Bug fix, no API change | PATCH | `1.2.3 β†’ 1.2.4` | +| New pre-release | suffix | `2.0.0a1`, `2.0.0rc1` | + +### Version in code β€” read from package metadata + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Never hardcode `__version__ = "1.0.0"` when using setuptools_scm β€” it goes stale after the +first git tag. Use `importlib.metadata` always. + +### Version specifier best practices for dependencies + +```toml +# In [project] dependencies β€” for a LIBRARY: +"httpx>=0.24" # Minimum version β€” PREFERRED for libraries +"httpx>=0.24,<1.0" # Upper bound only when a known breaking change exists + +# ONLY for applications (never for libraries): +"httpx==0.27.0" # Pin exactly β€” breaks dep resolution in libraries + +# NEVER do this in a library: +# "httpx~=0.24.0" # Compatible release operator β€” too tight +# "httpx==0.27.*" # Wildcard pin β€” fragile +``` + +--- + +## 6. Dynamic Versioning with `setuptools_scm` + +`setuptools_scm` reads your git tags and sets the package version automatically β€” no more manually +editing version strings before each release. + +### How it works + +``` +git tag v1.0.0 β†’ package version = 1.0.0 +git tag v1.1.0 β†’ package version = 1.1.0 +(commits after tag) β†’ version = 1.1.0.post1+g (stripped for PyPI) +``` + +`local_scheme = "no-local-version"` strips the `+g` suffix so PyPI uploads never fail with +a "local version label not allowed" error. + +### Access version at runtime + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0-dev" # Fallback for uninstalled dev checkouts +``` + +Never hardcode `__version__ = "1.0.0"` when using setuptools_scm β€” it will go stale after the +first tag. + +### Full release flow (this is it β€” nothing else needed) + +```bash +git tag v1.2.0 +git push origin master --tags +# GitHub Actions publish.yml triggers automatically +``` + +--- + +## 7. `setup.py` Shim + +Some older tools and IDEs still expect a `setup.py`. Keep it as a three-line shim β€” all real +configuration stays in `pyproject.toml`. + +```python +# setup.py β€” thin shim only. All config lives in pyproject.toml. +from setuptools import setup + +setup() +``` + +Never duplicate `name`, `version`, `dependencies`, or any other metadata from `pyproject.toml` +into `setup.py`. If you copy anything there it will eventually drift and cause confusing conflicts. + +--- + +## 8. Typed Package (PEP 561) + +A properly declared typed package means mypy, pyright, and IDEs automatically pick up your type +hints without any extra configuration from your users. + +### Step 1: Create the marker file + +```bash +# The file must exist; its content doesn't matter β€” its presence is the signal. +touch your_package/py.typed +``` + +### Step 2: Include it in the wheel + +Already in the template above: + +```toml +[tool.setuptools.package-data] +your_package = ["py.typed"] +``` + +### Step 3: Add the PyPI classifier + +```toml +classifiers = [ + ... + "Typing :: Typed", +] +``` + +### Step 4: Type-annotate all public functions + +```python +# Good β€” fully typed +def process( + self, + data: dict[str, object], + *, + timeout: int = 30, +) -> dict[str, object]: + ... + +# Bad β€” mypy will flag this, and IDEs give no completions to users +def process(self, data, timeout=30): + ... +``` + +### Step 5: Verify py.typed ships in the wheel + +```bash +python -m build +unzip -l dist/your_package-*.whl | grep py.typed +# Must show: your_package/py.typed +``` + +If it's missing, check your `[tool.setuptools.package-data]` config. diff --git a/skills/python-pypi-package-builder/references/release-governance.md b/skills/python-pypi-package-builder/references/release-governance.md new file mode 100644 index 00000000..5df7c4e5 --- /dev/null +++ b/skills/python-pypi-package-builder/references/release-governance.md @@ -0,0 +1,354 @@ +# Release Governance β€” Branching, Protection, OIDC, and Access Control + +## Table of Contents +1. [Branch Strategy](#1-branch-strategy) +2. [Branch Protection Rules](#2-branch-protection-rules) +3. [Tag-Based Release Model](#3-tag-based-release-model) +4. [Role-Based Access Control](#4-role-based-access-control) +5. [Secure Publishing with OIDC (Trusted Publishing)](#5-secure-publishing-with-oidc-trusted-publishing) +6. [Validate Tag Author in CI](#6-validate-tag-author-in-ci) +7. [Prevent Invalid Release Tags](#7-prevent-invalid-release-tags) +8. [Full `publish.yml` with Governance Gates](#8-full-publishyml-with-governance-gates) + +--- + +## 1. Branch Strategy + +Use a clear branch hierarchy to separate development work from releasable code. + +``` +main ← stable; only receives PRs from develop or hotfix/* +develop ← integration branch; all feature PRs merge here first +feature/* ← new capabilities (e.g., feature/add-redis-backend) +fix/* ← bug fixes (e.g., fix/memory-leak-on-close) +hotfix/* ← urgent production fixes; PR directly to main + cherry-pick to develop +release/* ← (optional) release preparation (e.g., release/v2.0.0) +``` + +### Rules + +| Rule | Why | +|---|---| +| No direct push to `main` | Prevent accidental breakage of the stable branch | +| All changes via PR | Enforces review + CI before merge | +| At least one approval required | Second pair of eyes on all changes | +| CI must pass | Never merge broken code | +| Only tags trigger releases | No ad-hoc publish from branch pushes | + +--- + +## 2. Branch Protection Rules + +Configure these in **GitHub β†’ Settings β†’ Branches β†’ Add rule** for `main` and `develop`. + +### For `main` + +```yaml +# Equivalent GitHub branch protection config (for documentation) +branch: main +rules: + - require_pull_request_reviews: + required_approving_review_count: 1 + dismiss_stale_reviews: true + - require_status_checks_to_pass: + contexts: + - "Lint, Format & Type Check" + - "Test (Python 3.11)" # at minimum; add all matrix versions + strict: true # branch must be up-to-date before merge + - restrict_pushes: + allowed_actors: [] # nobody β€” only PR merges + - require_linear_history: true # prevents merge commits on main +``` + +### For `develop` + +```yaml +branch: develop +rules: + - require_pull_request_reviews: + required_approving_review_count: 1 + - require_status_checks_to_pass: + contexts: ["CI"] + strict: false # less strict for the integration branch +``` + +### Via GitHub CLI + +```bash +# Protect main (requires gh CLI and admin rights) +gh api repos/{owner}/{repo}/branches/main/protection \ + --method PUT \ + --input - <<'EOF' +{ + "required_status_checks": { + "strict": true, + "contexts": ["Lint, Format & Type Check", "Test (Python 3.11)"] + }, + "enforce_admins": false, + "required_pull_request_reviews": { + "required_approving_review_count": 1, + "dismiss_stale_reviews": true + }, + "restrictions": null +} +EOF +``` + +--- + +## 3. Tag-Based Release Model + +**Only annotated tags on `main` trigger a release.** Branch pushes and PR merges never publish. + +### Tag Naming Convention + +``` +vMAJOR.MINOR.PATCH # Stable: v1.2.3 +vMAJOR.MINOR.PATCHaN # Alpha: v2.0.0a1 +vMAJOR.MINOR.PATCHbN # Beta: v2.0.0b1 +vMAJOR.MINOR.PATCHrcN # Release Candidate: v2.0.0rc1 +``` + +### Release Workflow + +```bash +# 1. Merge develop β†’ main via PR (reviewed, CI green) + +# 2. Update CHANGELOG.md on main +# Move [Unreleased] entries to [vX.Y.Z] - YYYY-MM-DD + +# 3. Commit the changelog +git checkout main +git pull origin main +git add CHANGELOG.md +git commit -m "chore: release v1.2.3" + +# 4. Create and push an annotated tag +git tag -a v1.2.3 -m "Release v1.2.3" +git push origin v1.2.3 # ← ONLY the tag; not --tags (avoids pushing all tags) + +# 5. Confirm: GitHub Actions publish.yml triggers automatically +# Monitor: Actions tab β†’ publish workflow +# Verify: https://pypi.org/project/your-package/ +``` + +### Why annotated tags? + +Annotated tags (`git tag -a`) carry a tagger identity, date, and message β€” lightweight tags do +not. `setuptools_scm` works with both, but annotated tags are safer for release governance because +they record *who* created the tag. + +--- + +## 4. Role-Based Access Control + +| Role | What they can do | +|---|---| +| **Maintainer** | Create release tags, approve PRs, manage branch protection | +| **Contributor** | Open PRs to `develop`; cannot push to `main` or create release tags | +| **CI (GitHub Actions)** | Publish to PyPI via OIDC; cannot push code or create tags | + +### Implement via GitHub Teams + +```bash +# Create a Maintainers team and restrict tag creation to that team +gh api repos/{owner}/{repo}/tags/protection \ + --method POST \ + --field pattern="v*" +# Then set allowed actors to the Maintainers team only +``` + +--- + +## 5. Secure Publishing with OIDC (Trusted Publishing) + +**Never store a PyPI API token as a GitHub secret.** Use Trusted Publishing (OIDC) instead. +The PyPI project authorises a specific GitHub repository + workflow + environment β€” no long-lived +secret is exchanged. + +### One-time PyPI Setup + +1. Go to https://pypi.org/manage/project/your-package/settings/publishing/ +2. Click **Add a new publisher** +3. Fill in: + - **Owner:** your-github-username + - **Repository:** your-repo-name + - **Workflow name:** `publish.yml` + - **Environment name:** `release` (must match the `environment:` key in the workflow) +4. Save. No token required. + +### GitHub Environment Setup + +1. Go to **GitHub β†’ Settings β†’ Environments β†’ New environment** β†’ name it `release` +2. Add a protection rule: **Required reviewers** (optional but recommended for extra safety) +3. Add a deployment branch rule: **Only tags matching `v*`** + +### Minimal `publish.yml` using OIDC + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v[0-9]+.[0-9]+.[0-9]+*" # Matches v1.0.0, v2.0.0a1, v1.2.3rc1 + +jobs: + publish: + name: Build and publish + runs-on: ubuntu-latest + environment: release # Must match the PyPI Trusted Publisher environment name + permissions: + id-token: write # Required for OIDC β€” grants a short-lived token to PyPI + contents: read + + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED for setuptools_scm + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build + run: pip install build + + - name: Build distributions + run: python -m build + + - name: Validate distributions + run: pip install twine ; twine check dist/* + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 + # No `password:` or `user:` needed β€” OIDC handles authentication +``` + +--- + +## 6. Validate Tag Author in CI + +Restrict who can trigger a release by checking `GITHUB_ACTOR` against an allowlist. +Add this as the **first step** in your publish job to fail fast. + +```yaml +- name: Validate tag author + run: | + ALLOWED_USERS=("your-github-username" "co-maintainer-username") + if [[ ! " ${ALLOWED_USERS[*]} " =~ " ${GITHUB_ACTOR} " ]]; then + echo "::error::Release blocked: ${GITHUB_ACTOR} is not an authorised releaser." + exit 1 + fi + echo "Release authorised for ${GITHUB_ACTOR}." +``` + +### Notes + +- `GITHUB_ACTOR` is the GitHub username of the person who pushed the tag. +- Store the allowlist in a separate file (e.g., `.github/MAINTAINERS`) for maintainability. +- For teams: replace the username check with a GitHub API call to verify team membership. + +--- + +## 7. Prevent Invalid Release Tags + +Reject workflow runs triggered by tags that do not follow your versioning convention. +This stops accidental publishes from tags like `test`, `backup-old`, or `v1`. + +```yaml +- name: Validate release tag format + run: | + # Accepts: v1.0.0 v1.0.0a1 v1.0.0b2 v1.0.0rc1 v1.0.0.post1 + if [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(a|b|rc|\.post)[0-9]*$ ]] && \ + [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "::error::Tag '${GITHUB_REF}' does not match the required format v..[pre]." + exit 1 + fi + echo "Tag format valid: ${GITHUB_REF}" +``` + +### Regex explained + +| Pattern | Matches | +|---|---| +| `v[0-9]+\.[0-9]+\.[0-9]+` | `v1.0.0`, `v12.3.4` | +| `(a\|b\|rc)[0-9]*` | `v1.0.0a1`, `v2.0.0rc2` | +| `\.post[0-9]*` | `v1.0.0.post1` | + +--- + +## 8. Full `publish.yml` with Governance Gates + +Complete workflow combining tag validation, author check, TestPyPI gate, and production publish. + +```yaml +# .github/workflows/publish.yml +name: Publish to PyPI + +on: + push: + tags: + - "v[0-9]+.[0-9]+.[0-9]+*" + +jobs: + publish: + name: Build, validate, and publish + runs-on: ubuntu-latest + environment: release + permissions: + id-token: write + contents: read + + steps: + - name: Validate release tag format + run: | + if [[ ! "${GITHUB_REF}" =~ ^refs/tags/v[0-9]+\.[0-9]+\.[0-9]+(a[0-9]*|b[0-9]*|rc[0-9]*|\.post[0-9]*)?$ ]]; then + echo "::error::Invalid tag format: ${GITHUB_REF}" + exit 1 + fi + + - name: Validate tag author + run: | + ALLOWED_USERS=("your-github-username") + if [[ ! " ${ALLOWED_USERS[*]} " =~ " ${GITHUB_ACTOR} " ]]; then + echo "::error::${GITHUB_ACTOR} is not authorised to release." + exit 1 + fi + + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install build tooling + run: pip install build twine + + - name: Build + run: python -m build + + - name: Validate distributions + run: twine check dist/* + + - name: Publish to TestPyPI + uses: pypa/gh-action-pypi-publish@release/v1 + with: + repository-url: https://test.pypi.org/legacy/ + continue-on-error: true # Non-fatal; remove if you always want this to pass + + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +``` + +### Security checklist + +- [ ] PyPI Trusted Publishing configured (no API token stored in GitHub) +- [ ] GitHub `release` environment has branch protection: tags matching `v*` only +- [ ] Tag format validation step is the first step in the job +- [ ] Allowed-users list is maintained and reviewed regularly +- [ ] No secrets printed in logs (check all `echo` and `run` steps) +- [ ] `permissions:` is scoped to `id-token: write` only β€” no `write-all` diff --git a/skills/python-pypi-package-builder/references/testing-quality.md b/skills/python-pypi-package-builder/references/testing-quality.md new file mode 100644 index 00000000..b6fbbec2 --- /dev/null +++ b/skills/python-pypi-package-builder/references/testing-quality.md @@ -0,0 +1,257 @@ +# Testing and Code Quality + +## Table of Contents +1. [conftest.py](#1-conftestpy) +2. [Unit tests](#2-unit-tests) +3. [Backend unit tests](#3-backend-unit-tests) +4. [Running tests](#4-running-tests) +5. [Code quality tools](#5-code-quality-tools) +6. [Pre-commit hooks](#6-pre-commit-hooks) + +--- + +## 1. `conftest.py` + +Use `conftest.py` to define shared fixtures. Keep fixtures focused β€” one fixture per concern. +For async tests, use `pytest-asyncio` with `asyncio_mode = "auto"` in `pyproject.toml`. + +```python +# tests/conftest.py +import pytest +from your_package.core import YourClient +from your_package.backends.memory import MemoryBackend + + +@pytest.fixture +def memory_backend() -> MemoryBackend: + return MemoryBackend() + + +@pytest.fixture +def client(memory_backend: MemoryBackend) -> YourClient: + return YourClient( + api_key="test-key", + backend=memory_backend, + ) +``` + +--- + +## 2. Unit Tests + +Test both the happy path and the edge cases (e.g. invalid inputs, error conditions). + +```python +# tests/test_core.py +import pytest +from your_package import YourClient +from your_package.exceptions import YourPackageError + + +def test_client_creates_with_valid_key(): + client = YourClient(api_key="sk-test") + assert client is not None + + +def test_client_raises_on_empty_key(): + with pytest.raises(ValueError, match="api_key"): + YourClient(api_key="") + + +def test_client_raises_on_invalid_timeout(): + with pytest.raises(ValueError, match="timeout"): + YourClient(api_key="sk-test", timeout=-1) + + +@pytest.mark.asyncio +async def test_process_returns_expected_result(client: YourClient): + result = await client.process({"input": "value"}) + assert "output" in result + + +@pytest.mark.asyncio +async def test_process_raises_on_invalid_input(client: YourClient): + with pytest.raises(YourPackageError): + await client.process({}) # empty input should fail +``` + +--- + +## 3. Backend Unit Tests + +Test each backend independently, in isolation from the rest of the library. This makes failures +easier to diagnose and ensures your abstract interface is actually implemented correctly. + +```python +# tests/test_backends.py +import pytest +from your_package.backends.memory import MemoryBackend + + +@pytest.mark.asyncio +async def test_set_and_get(): + backend = MemoryBackend() + await backend.set("key1", "value1") + result = await backend.get("key1") + assert result == "value1" + + +@pytest.mark.asyncio +async def test_get_missing_key_returns_none(): + backend = MemoryBackend() + result = await backend.get("nonexistent") + assert result is None + + +@pytest.mark.asyncio +async def test_delete_removes_key(): + backend = MemoryBackend() + await backend.set("key1", "value1") + await backend.delete("key1") + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_ttl_expires_entry(): + import asyncio + backend = MemoryBackend() + await backend.set("key1", "value1", ttl=1) + await asyncio.sleep(1.1) + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_different_keys_are_independent(): + backend = MemoryBackend() + await backend.set("key1", "a") + await backend.set("key2", "b") + assert await backend.get("key1") == "a" + assert await backend.get("key2") == "b" + await backend.delete("key1") + assert await backend.get("key2") == "b" +``` + +--- + +## 4. Running Tests + +```bash +pip install -e ".[dev]" +pytest # All tests +pytest --cov --cov-report=html # With HTML coverage report (opens in browser) +pytest -k "test_middleware" # Filter by name +pytest -x # Stop on first failure +pytest -v # Verbose output +``` + +Coverage config in `pyproject.toml` enforces a minimum threshold (`fail_under = 80`). CI will +fail if you drop below it, which catches coverage regressions automatically. + +--- + +## 5. Code Quality Tools + +### Ruff (linting β€” replaces flake8, pylint, many others) + +```bash +pip install ruff +ruff check . # Check for issues +ruff check . --fix # Auto-fix safe issues +``` + +Ruff is extremely fast and replaces most of the Python linting ecosystem. Configure it in +`pyproject.toml` β€” see `references/pyproject-toml.md` for the full config. + +### Black (formatting) + +```bash +pip install black +black . # Format all files +black . --check # CI mode β€” reports issues without modifying files +``` + +### isort (import sorting) + +```bash +pip install isort +isort . # Sort imports +isort . --check-only # CI mode +``` + +Always set `profile = "black"` in `[tool.isort]` β€” otherwise black and isort conflict. + +### mypy (static type checking) + +```bash +pip install mypy +mypy your_package/ # Type-check your package source only +``` + +Common fixes: + +- `ignore_missing_imports = true` β€” ignore untyped third-party deps +- `from __future__ import annotations` β€” enables PEP 563 deferred evaluation (Python 3.9 compat) +- `pip install types-redis` β€” type stubs for the redis library + +### Run all at once + +```bash +ruff check . && black . --check && isort . --check-only && mypy your_package/ +``` + +--- + +## 6. Pre-commit Hooks + +Pre-commit runs all quality tools automatically before each commit, so issues never reach CI. +Install once per clone with `pre-commit install`. + +```yaml +# .pre-commit-config.yaml +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 + hooks: + - id: ruff + args: [--fix] + - id: ruff-format + + - repo: https://github.com/psf/black + rev: 24.4.2 + hooks: + - id: black + + - repo: https://github.com/pycqa/isort + rev: 5.13.2 + hooks: + - id: isort + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + additional_dependencies: [types-redis] # Add stubs for typed dependencies + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-toml + - id: check-merge-conflict + - id: debug-statements + - id: no-commit-to-branch + args: [--branch, master, --branch, main] +``` + +```bash +pip install pre-commit +pre-commit install # Install once per clone +pre-commit run --all-files # Run all hooks manually (useful before the first install) +``` + +The `no-commit-to-branch` hook prevents accidentally committing directly to `main`/`master`, +which would bypass CI checks. Always work on a feature branch. diff --git a/skills/python-pypi-package-builder/references/tooling-ruff.md b/skills/python-pypi-package-builder/references/tooling-ruff.md new file mode 100644 index 00000000..1d3cc27a --- /dev/null +++ b/skills/python-pypi-package-builder/references/tooling-ruff.md @@ -0,0 +1,344 @@ +# Tooling β€” Ruff-Only Setup and Code Quality + +## Table of Contents +1. [Use Only Ruff (Replaces black, isort, flake8)](#1-use-only-ruff-replaces-black-isort-flake8) +2. [Ruff Configuration in pyproject.toml](#2-ruff-configuration-in-pyprojecttoml) +3. [mypy Configuration](#3-mypy-configuration) +4. [pre-commit Configuration](#4-pre-commit-configuration) +5. [pytest and Coverage Configuration](#5-pytest-and-coverage-configuration) +6. [Dev Dependencies in pyproject.toml](#6-dev-dependencies-in-pyprojecttoml) +7. [CI Lint Job β€” Ruff Only](#7-ci-lint-job--ruff-only) +8. [Migration Guide β€” Removing black and isort](#8-migration-guide--removing-black-and-isort) + +--- + +## 1. Use Only Ruff (Replaces black, isort, flake8) + +**Decision:** Use `ruff` as the single linting and formatting tool. Remove `black` and `isort`. + +| Old (avoid) | New (use) | What it does | +|---|---|---| +| `black` | `ruff format` | Code formatting | +| `isort` | `ruff check --select I` | Import sorting | +| `flake8` | `ruff check` | Style and error linting | +| `pyupgrade` | `ruff check --select UP` | Upgrade syntax to modern Python | +| `bandit` | `ruff check --select S` | Security linting | +| All of the above | `ruff` | One tool, one config section | + +**Why ruff?** +- 10–100Γ— faster than the tools it replaces (written in Rust). +- Single config section in `pyproject.toml` β€” no `.flake8`, `.isort.cfg`, `pyproject.toml[tool.black]` sprawl. +- Actively maintained by Astral; follows the same rules as the tools it replaces. +- `ruff format` is black-compatible β€” existing black-formatted code passes without changes. + +--- + +## 2. Ruff Configuration in pyproject.toml + +```toml +[tool.ruff] +target-version = "py310" # Minimum supported Python version +line-length = 88 # black-compatible default +src = ["src", "tests"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # pyflakes + "I", # isort + "B", # flake8-bugbear (opinionated but very useful) + "C4", # flake8-comprehensions + "UP", # pyupgrade (modernise syntax) + "SIM", # flake8-simplify + "TCH", # flake8-type-checking (move imports to TYPE_CHECKING block) + "ANN", # flake8-annotations (enforce type hints β€” remove if too strict) + "S", # flake8-bandit (security) + "N", # pep8-naming +] +ignore = [ + "ANN101", # Missing type annotation for `self` + "ANN102", # Missing type annotation for `cls` + "S101", # Use of `assert` β€” necessary in tests + "S603", # subprocess without shell=True β€” often intentional + "B008", # Do not perform function calls in default arguments (false positives in FastAPI/Typer) +] + +[tool.ruff.lint.isort] +known-first-party = ["your_package"] + +[tool.ruff.lint.per-file-ignores] +"tests/**" = ["S101", "ANN", "D"] # Allow assert and skip annotations/docstrings in tests + +[tool.ruff.format] +quote-style = "double" # black-compatible +indent-style = "space" +skip-magic-trailing-comma = false +line-ending = "auto" +``` + +### Useful ruff commands + +```bash +# Check for lint issues (no changes) +ruff check . + +# Auto-fix fixable issues +ruff check --fix . + +# Format code (replaces black) +ruff format . + +# Check formatting without changing files (CI mode) +ruff format --check . + +# Run both lint and format check in one command (for CI) +ruff check . && ruff format --check . +``` + +--- + +## 3. mypy Configuration + +```toml +[tool.mypy] +python_version = "3.10" +strict = true +warn_return_any = true +warn_unused_ignores = true +warn_redundant_casts = true +disallow_untyped_defs = true +disallow_incomplete_defs = true +check_untyped_defs = true +no_implicit_optional = true +show_error_codes = true + +# Ignore missing stubs for third-party packages that don't ship types +[[tool.mypy.overrides]] +module = ["redis.*", "pydantic_settings.*"] +ignore_missing_imports = true +``` + +### Running mypy β€” handle both src and flat layouts + +```bash +# src layout: +mypy src/your_package/ + +# flat layout: +mypy your_package/ +``` + +In CI, detect layout dynamically: + +```yaml +- name: Run mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy your_package/ + fi +``` + +--- + +## 4. pre-commit Configuration + +```yaml +# .pre-commit-config.yaml +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 # Pin to a specific release; update periodically with `pre-commit autoupdate` + hooks: + - id: ruff + args: [--fix] # Auto-fix what can be fixed + - id: ruff-format # Format (replaces black hook) + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + additional_dependencies: + - types-requests + - types-redis + # Add stubs for any typed dependency used in your package + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-toml + - id: check-yaml + - id: check-merge-conflict + - id: check-added-large-files + args: ["--maxkb=500"] +``` + +### ❌ Remove these hooks (replaced by ruff) + +```yaml +# DELETE or never add: +- repo: https://github.com/psf/black # replaced by ruff-format +- repo: https://github.com/PyCQA/isort # replaced by ruff lint I rules +- repo: https://github.com/PyCQA/flake8 # replaced by ruff check +- repo: https://github.com/PyCQA/autoflake # replaced by ruff check F401 +``` + +### Setup + +```bash +pip install pre-commit +pre-commit install # Installs git hook β€” runs on every commit +pre-commit run --all-files # Run manually on all files +pre-commit autoupdate # Update all hooks to latest pinned versions +``` + +--- + +## 5. pytest and Coverage Configuration + +```toml +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-ra -q --strict-markers --cov=your_package --cov-report=term-missing" +asyncio_mode = "auto" # Enables async tests without @pytest.mark.asyncio decorator + +[tool.coverage.run] +source = ["your_package"] +branch = true +omit = ["**/__main__.py", "**/cli.py"] # omit entry points from coverage + +[tool.coverage.report] +show_missing = true +skip_covered = false +fail_under = 85 # Fail CI if coverage drops below 85% +exclude_lines = [ + "pragma: no cover", + "if TYPE_CHECKING:", + "raise NotImplementedError", + "@abstractmethod", +] +``` + +### asyncio_mode = "auto" β€” remove @pytest.mark.asyncio + +With `asyncio_mode = "auto"` set in `pyproject.toml`, **do not** add `@pytest.mark.asyncio` +to test functions. The decorator is redundant and will raise a warning in modern pytest-asyncio. + +```python +# WRONG β€” the decorator is deprecated when asyncio_mode = "auto": +@pytest.mark.asyncio +async def test_async_operation(): + result = await my_async_func() + assert result == expected + +# CORRECT β€” just use async def: +async def test_async_operation(): + result = await my_async_func() + assert result == expected +``` + +--- + +## 6. Dev Dependencies in pyproject.toml + +Declare all dev/test tools in an `[extras]` group named `dev`. + +```toml +[project.optional-dependencies] +dev = [ + "pytest>=8", + "pytest-asyncio>=0.23", + "pytest-cov>=5", + "ruff>=0.4", + "mypy>=1.10", + "pre-commit>=3.7", + "httpx>=0.27", # If testing HTTP transport + "respx>=0.21", # If mocking httpx in tests +] +redis = [ + "redis>=5", +] +docs = [ + "mkdocs-material>=9", + "mkdocstrings[python]>=0.25", +] +``` + +Install dev dependencies: + +```bash +pip install -e ".[dev]" +pip install -e ".[dev,redis]" # Include optional extras +``` + +--- + +## 7. CI Lint Job β€” Ruff Only + +Replace the separate `black`, `isort`, and `flake8` steps with a single `ruff` step. + +```yaml +# .github/workflows/ci.yml β€” lint job +lint: + name: Lint & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + + - name: Install dev dependencies + run: pip install -e ".[dev]" + + # Single step: ruff replaces black + isort + flake8 + - name: ruff lint + run: ruff check . + + - name: ruff format check + run: ruff format --check . + + - name: mypy + run: | + if [ -d "src" ]; then + mypy src/ + else + mypy $(basename $(ls -d */))/ 2>/dev/null || mypy . + fi +``` + +--- + +## 8. Migration Guide β€” Removing black and isort + +If you are converting an existing project that used `black` and `isort`: + +```bash +# 1. Remove black and isort from dev dependencies +pip uninstall black isort + +# 2. Remove black and isort config sections from pyproject.toml +# [tool.black] ← delete this section +# [tool.isort] ← delete this section + +# 3. Add ruff to dev dependencies (see Section 2 for config) + +# 4. Run ruff format to confirm existing code is already compatible +ruff format --check . +# ruff format is black-compatible; output should be identical + +# 5. Update .pre-commit-config.yaml (see Section 4) +# Remove black and isort hooks; add ruff and ruff-format hooks + +# 6. Update CI (see Section 7) +# Remove black, isort, flake8 steps; add ruff check + ruff format --check + +# 7. Reinstall pre-commit hooks +pre-commit uninstall +pre-commit install +pre-commit run --all-files # Verify clean +``` diff --git a/skills/python-pypi-package-builder/references/versioning-strategy.md b/skills/python-pypi-package-builder/references/versioning-strategy.md new file mode 100644 index 00000000..5f49ad8c --- /dev/null +++ b/skills/python-pypi-package-builder/references/versioning-strategy.md @@ -0,0 +1,375 @@ +# Versioning Strategy β€” PEP 440, SemVer, and Decision Engine + +## Table of Contents +1. [PEP 440 β€” The Standard](#1-pep-440--the-standard) +2. [Semantic Versioning (SemVer)](#2-semantic-versioning-semver) +3. [Pre-release Identifiers](#3-pre-release-identifiers) +4. [Versioning Decision Engine](#4-versioning-decision-engine) +5. [Dynamic Versioning β€” setuptools_scm (Recommended)](#5-dynamic-versioning--setuptools_scm-recommended) +6. [Hatchling with hatch-vcs Plugin](#6-hatchling-with-hatch-vcs-plugin) +7. [Static Versioning β€” flit](#7-static-versioning--flit) +8. [Static Versioning β€” hatchling manual](#8-static-versioning--hatchling-manual) +9. [DO NOT Hardcode Version (except flit)](#9-do-not-hardcode-version-except-flit) +10. [Dependency Version Specifiers](#10-dependency-version-specifiers) +11. [PyPA Release Commands](#11-pypa-release-commands) + +--- + +## 1. PEP 440 β€” The Standard + +All Python package versions must comply with [PEP 440](https://peps.python.org/pep-0440/). +Non-compliant versions (e.g., `1.0-beta`, `2023.1.1.dev`) will be rejected by PyPI. + +``` +Canonical form: N[.N]+[{a|b|rc}N][.postN][.devN] + +1.0.0 Stable release +1.0.0a1 Alpha pre-release +1.0.0b2 Beta pre-release +1.0.0rc1 Release candidate +1.0.0.post1 Post-release (packaging fix; same codebase) +1.0.0.dev1 Development snapshot β€” DO NOT upload to PyPI +2.0.0 Major release (breaking changes) +``` + +### Epoch prefix (rare) + +``` +1!1.0.0 Epoch 1; used when you need to skip ahead of an old scheme +``` + +Use epochs only as a last resort to fix a broken version sequence. + +--- + +## 2. Semantic Versioning (SemVer) + +SemVer maps cleanly onto PEP 440. Always use `MAJOR.MINOR.PATCH`: + +``` +MAJOR Increment when you make incompatible API changes (rename, remove, break) +MINOR Increment when you add functionality backward-compatibly (new features) +PATCH Increment when you make backward-compatible bug fixes + +Examples: + 1.0.0 β†’ 1.0.1 Bug fix, no API change + 1.0.0 β†’ 1.1.0 New method added; existing API intact + 1.0.0 β†’ 2.0.0 Public method renamed or removed +``` + +### What counts as a breaking change? + +| Change | Breaking? | +|---|---| +| Rename a public function | YES β€” `MAJOR` | +| Remove a parameter | YES β€” `MAJOR` | +| Add a required parameter | YES β€” `MAJOR` | +| Add an optional parameter with a default | NO β€” `MINOR` | +| Add a new function/class | NO β€” `MINOR` | +| Fix a bug | NO β€” `PATCH` | +| Update a dependency lower bound | NO (usually) β€” `PATCH` | +| Update a dependency upper bound (breaking) | YES β€” `MAJOR` | + +--- + +## 3. Pre-release Identifiers + +Use pre-release versions to get user feedback before a stable release. +Pre-releases are **not** installed by default by pip (`pip install pkg` skips them). +Users must opt-in: `pip install "pkg==2.0.0a1"` or `pip install --pre pkg`. + +``` +1.0.0a1 Alpha-1: very early; expect bugs; API may change +1.0.0b1 Beta-1: feature-complete; API stabilising; seek broader feedback +1.0.0rc1 Release candidate: code-frozen; final testing before stable +1.0.0 Stable: ready for production +``` + +### Increment rule + +``` +Start: 1.0.0a1 +More alphas: 1.0.0a2, 1.0.0a3 +Move to beta: 1.0.0b1 (reset counter) +Move to RC: 1.0.0rc1 +Stable: 1.0.0 +``` + +--- + +## 4. Versioning Decision Engine + +Use this decision tree to pick the right versioning strategy before writing any code. + +``` +Is the project using git and tagging releases with version tags? +β”œβ”€β”€ YES β†’ setuptools + setuptools_scm (DEFAULT β€” best for most projects) +β”‚ Git tag v1.0.0 becomes the installed version automatically. +β”‚ Zero manual version bumping. +β”‚ +└── NO β€” Is the project a simple, single-module library with infrequent releases? + β”œβ”€β”€ YES β†’ flit + β”‚ Set __version__ = "1.0.0" in __init__.py. + β”‚ Update manually before each release. + β”‚ + └── NO β€” Does the team want an integrated build + dep management tool? + β”œβ”€β”€ YES β†’ poetry + β”‚ Manage version in [tool.poetry] version field. + β”‚ + └── NO β†’ hatchling (modern, fast, pure-Python) + Use hatch-vcs plugin for dynamic versioning + or set version manually in [project]. + +Does the package have C/Cython/Fortran extensions? +└── YES (always) β†’ setuptools (only backend with native extension support) +``` + +### Summary Table + +| Backend | Version source | Best for | +|---|---|---| +| `setuptools` + `setuptools_scm` | Git tags β€” fully automatic | DEFAULT for new projects | +| `hatchling` + `hatch-vcs` | Git tags β€” automatic via plugin | hatchling users | +| `flit` | `__version__` in `__init__.py` | Very simple, minimal config | +| `poetry` | `[tool.poetry] version` field | Integrated dep + build management | +| `hatchling` manual | `[project] version` field | One-off static versioning | + +--- + +## 5. Dynamic Versioning β€” setuptools_scm (Recommended) + +`setuptools_scm` reads the current git tag and computes the version at build time. +No separate `__version__` update step β€” just tag and push. + +### `pyproject.toml` configuration + +```toml +[build-system] +requires = ["setuptools>=70", "setuptools_scm>=8"] +build-backend = "setuptools.backends.legacy:build" + +[project] +name = "your-package" +dynamic = ["version"] + +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" # Prevents +g from breaking PyPI +``` + +### `__init__.py` β€” correct version access + +```python +# your_package/__init__.py +from importlib.metadata import version, PackageNotFoundError + +try: + __version__ = version("your-package") +except PackageNotFoundError: + # Package is not installed (running from a source checkout without pip install -e .) + __version__ = "0.0.0.dev0" + +__all__ = ["__version__"] +``` + +### How the version is computed + +``` +git tag v1.0.0 β†’ installed_version = "1.0.0" +3 commits after v1.0.0 β†’ installed_version = "1.0.0.post3+g" (dev only) +git tag v1.1.0 β†’ installed_version = "1.1.0" +``` + +With `local_scheme = "no-local-version"`, the `+g` suffix is stripped for PyPI +uploads while still being visible locally. + +### Critical CI requirement + +```yaml +- uses: actions/checkout@v4 + with: + fetch-depth: 0 # REQUIRED β€” without this, git has no tag history + # setuptools_scm falls back to 0.0.0+d silently +``` + +**Every** CI job that installs or builds the package must have `fetch-depth: 0`. + +### Debugging version issues + +```bash +# Check what version setuptools_scm would produce right now: +python -m setuptools_scm + +# If you see 0.0.0+d... it means: +# 1. No tags reachable from HEAD, OR +# 2. fetch-depth: 0 was not set in CI +``` + +--- + +## 6. Hatchling with hatch-vcs Plugin + +An alternative to setuptools_scm for teams already using hatchling. + +```toml +[build-system] +requires = ["hatchling", "hatch-vcs"] +build-backend = "hatchling.build" + +[project] +name = "your-package" +dynamic = ["version"] + +[tool.hatch.version] +source = "vcs" + +[tool.hatch.build.hooks.vcs] +version-file = "src/your_package/_version.py" +``` + +Access the version the same way as setuptools_scm: + +```python +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0.dev0" +``` + +--- + +## 7. Static Versioning β€” flit + +Use flit only for simple, single-module packages where manual version bumping is acceptable. + +### `pyproject.toml` + +```toml +[build-system] +requires = ["flit_core>=3.9"] +build-backend = "flit_core.buildapi" + +[project] +name = "your-package" +dynamic = ["version", "description"] +``` + +### `__init__.py` + +```python +"""your-package β€” a focused, single-purpose utility.""" +__version__ = "1.2.0" # flit reads this; update manually before each release +``` + +**flit exception:** this is the ONLY case where hardcoding `__version__` is correct. +flit discovers the version by importing `__init__.py` and reading `__version__`. + +### Release flow for flit + +```bash +# 1. Bump __version__ in __init__.py +# 2. Update CHANGELOG.md +# 3. Commit +git add src/your_package/__init__.py CHANGELOG.md +git commit -m "chore: release v1.2.0" +# 4. Tag (flit can also publish directly) +git tag v1.2.0 +git push origin v1.2.0 +# 5. Build and publish +flit publish +# OR +python -m build && twine upload dist/* +``` + +--- + +## 8. Static Versioning β€” hatchling manual + +```toml +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "your-package" +version = "1.0.0" # Manual; update before each release +``` + +Update `version` in `pyproject.toml` before every release. No `__version__` required +(access via `importlib.metadata.version()` as usual). + +--- + +## 9. DO NOT Hardcode Version (except flit) + +Hardcoding `__version__` in `__init__.py` when **not** using flit creates a dual source of +truth that diverges over time. + +```python +# BAD β€” when using setuptools_scm, hatchling, or poetry: +__version__ = "1.0.0" # gets stale; diverges from the installed package version + +# GOOD β€” works for all backends except flit: +from importlib.metadata import version, PackageNotFoundError +try: + __version__ = version("your-package") +except PackageNotFoundError: + __version__ = "0.0.0.dev0" +``` + +--- + +## 10. Dependency Version Specifiers + +Pick the right specifier style to avoid poisoning your users' environments. + +```toml +# [project] dependencies β€” library best practices: + +"httpx>=0.24" # Minimum only β€” PREFERRED; lets users upgrade freely +"httpx>=0.24,<2.0" # Upper bound only when a known breaking change exists in next major +"requests>=2.28,<3.0" # Acceptable for well-known major-version breaks + +# Application / CLI (pinning is fine): +"httpx==0.27.2" # Lock exact version for reproducible deploys + +# NEVER in a library: +# "httpx~=0.24.0" # Too tight; blocks minor upgrades +# "httpx==0.27.*" # Not valid PEP 440 +# "httpx" # No constraint; fragile against future breakage +``` + +--- + +## 11. PyPA Release Commands + +The canonical sequence from code to user install. + +```bash +# Step 1: Tag the release (triggers CI publish.yml automatically if configured) +git tag -a v1.2.3 -m "Release v1.2.3" +git push origin v1.2.3 + +# Step 2 (manual fallback only): Build locally +python -m build +# Produces: +# dist/your_package-1.2.3.tar.gz (sdist) +# dist/your_package-1.2.3-py3-none-any.whl (wheel) + +# Step 3: Validate +twine check dist/* + +# Step 4: Test on TestPyPI first (first release or major change) +twine upload --repository testpypi dist/* +pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ your-package==1.2.3 + +# Step 5: Publish to production PyPI +twine upload dist/* +# OR via GitHub Actions (recommended): +# push the tag β†’ publish.yml runs β†’ pypa/gh-action-pypi-publish handles upload via OIDC + +# Step 6: Verify +pip install your-package==1.2.3 +python -c "import your_package; print(your_package.__version__)" +``` diff --git a/skills/python-pypi-package-builder/scripts/scaffold.py b/skills/python-pypi-package-builder/scripts/scaffold.py new file mode 100644 index 00000000..4479d831 --- /dev/null +++ b/skills/python-pypi-package-builder/scripts/scaffold.py @@ -0,0 +1,920 @@ +#!/usr/bin/env python3 +""" +scaffold.py β€” Generate a production-grade Python PyPI package structure. + +Usage: + python scaffold.py --name my-package + python scaffold.py --name my-package --layout src + python scaffold.py --name my-package --build hatchling + +Options: + --name PyPI package name (lowercase, hyphens). Required. + --layout 'flat' (default) or 'src'. + --build 'setuptools' (default, uses setuptools_scm) or 'hatchling'. + --author Author name (default: Your Name). + --email Author email (default: you@example.com). + --output Output directory (default: current directory). +""" + +import argparse +import os +import sys +import textwrap +from pathlib import Path + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + +def pkg_name(pypi_name: str) -> str: + """Convert 'my-pkg' β†’ 'my_pkg'.""" + return pypi_name.replace("-", "_") + + +def write(path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(textwrap.dedent(content).lstrip(), encoding="utf-8") + print(f" created {path}") + + +def touch(path: Path) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.touch() + print(f" created {path}") + + +# --------------------------------------------------------------------------- +# File generators +# --------------------------------------------------------------------------- + +def gen_pyproject_setuptools(name: str, mod: str, author: str, email: str, layout: str) -> str: + packages_find = ( + 'where = ["src"]' if layout == "src" else f'include = ["{mod}*"]' + ) + pkg_data_key = f"src/{mod}" if layout == "src" else mod + pythonpath = "" if layout == "src" else '\npythonpath = ["."]' + return f'''\ +[build-system] +requires = ["setuptools>=68", "wheel", "setuptools_scm"] +build-backend = "setuptools.build_meta" + +[project] +name = "{name}" +dynamic = ["version"] +description = "" +readme = "README.md" +requires-python = ">=3.10" +license = {{text = "MIT"}} +authors = [ + {{name = "{author}", email = "{email}"}}, +] +keywords = [ + "python", + # Add 10-15 specific keywords β€” they affect PyPI discoverability +] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Operating System :: OS Independent", + "Programming Language :: Python :: 3", + "Programming Language :: Python :: 3.10", + "Programming Language :: Python :: 3.11", + "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", + "Topic :: Software Development :: Libraries :: Python Modules", + "Typing :: Typed", +] +dependencies = [ + # List your runtime dependencies here. Keep them minimal. + # Example: "httpx>=0.24", "pydantic>=2.0" +] +] + +[project.optional-dependencies] +redis = [ + "redis>=4.2", +] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/{name}" +Documentation = "https://github.com/yourusername/{name}#readme" +Repository = "https://github.com/yourusername/{name}" +"Bug Tracker" = "https://github.com/yourusername/{name}/issues" +Changelog = "https://github.com/yourusername/{name}/blob/master/CHANGELOG.md" + +[tool.setuptools.packages.find] +{packages_find} + +[tool.setuptools.package-data] +{mod} = ["py.typed"] + +[tool.setuptools_scm] +version_scheme = "post-release" +local_scheme = "no-local-version" + +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.ruff.lint] +select = ["E", "F", "W", "I", "N", "UP", "B", "SIM", "C4", "PTH"] + +[tool.ruff.lint.per-file-ignores] +"tests/*" = ["S101"] + +[tool.black] +line-length = 100 +target-version = ["py310", "py311", "py312", "py313"] + +[tool.isort] +profile = "black" +line_length = 100 + +[tool.mypy] +python_version = "3.10" +warn_return_any = true +warn_unused_configs = true +disallow_untyped_defs = true +ignore_missing_imports = true +strict = false + +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"]{pythonpath} +python_files = "test_*.py" +python_classes = "Test*" +python_functions = "test_*" +addopts = "-v --tb=short --cov={mod} --cov-report=term-missing" + +[tool.coverage.run] +source = ["{mod}"] +omit = ["tests/*"] + +[tool.coverage.report] +fail_under = 80 +show_missing = true +''' + + +def gen_pyproject_hatchling(name: str, mod: str, author: str, email: str) -> str: + return f'''\ +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "{name}" +version = "0.1.0" +description = "" +readme = "README.md" +requires-python = ">=3.10" +license = {{text = "MIT"}} +authors = [ + {{name = "{author}", email = "{email}"}}, +] +keywords = ["python"] +classifiers = [ + "Development Status :: 3 - Alpha", + "Intended Audience :: Developers", + "License :: OSI Approved :: MIT License", + "Programming Language :: Python :: 3", + "Typing :: Typed", +] +dependencies = [ + # List your runtime dependencies here. +] + +[project.optional-dependencies] +dev = [ + "pytest>=7.0", + "pytest-asyncio>=0.21", + "httpx>=0.24", + "pytest-cov>=4.0", + "ruff>=0.4", + "black>=24.0", + "isort>=5.13", + "mypy>=1.0", + "pre-commit>=3.0", + "build", + "twine", +] + +[project.urls] +Homepage = "https://github.com/yourusername/{name}" +Changelog = "https://github.com/yourusername/{name}/blob/master/CHANGELOG.md" + +[tool.hatch.build.targets.wheel] +packages = ["{mod}"] + +[tool.hatch.build.targets.wheel.sources] +"{mod}" = "{mod}" + +[tool.ruff] +target-version = "py310" +line-length = 100 + +[tool.black] +line-length = 100 + +[tool.isort] +profile = "black" + +[tool.mypy] +python_version = "3.10" +disallow_untyped_defs = true +ignore_missing_imports = true + +[tool.pytest.ini_options] +asyncio_mode = "auto" +testpaths = ["tests"] +addopts = "-v --tb=short --cov={mod} --cov-report=term-missing" + +[tool.coverage.report] +fail_under = 80 +show_missing = true +''' + + +def gen_init(name: str, mod: str) -> str: + return f'''\ +"""{name}: .""" + +from importlib.metadata import PackageNotFoundError, version + +try: + __version__ = version("{name}") +except PackageNotFoundError: + __version__ = "0.0.0-dev" + +from {mod}.core import YourClient +from {mod}.config import YourSettings +from {mod}.exceptions import YourPackageError + +__all__ = [ + "YourClient", + "YourSettings", + "YourPackageError", + "__version__", +] +''' + + +def gen_core(mod: str) -> str: + return f'''\ +from __future__ import annotations + +from {mod}.exceptions import YourPackageError + + +class YourClient: + """ + Main entry point for . + + Args: + api_key: Required authentication credential. + timeout: Request timeout in seconds. Defaults to 30. + retries: Number of retry attempts. Defaults to 3. + + Raises: + ValueError: If api_key is empty or timeout is non-positive. + + Example: + >>> from {mod} import YourClient + >>> client = YourClient(api_key="sk-...") + >>> result = client.process(data) + """ + + def __init__( + self, + api_key: str, + timeout: int = 30, + retries: int = 3, + ) -> None: + if not api_key: + raise ValueError("api_key must not be empty") + if timeout <= 0: + raise ValueError("timeout must be positive") + self._api_key = api_key + self.timeout = timeout + self.retries = retries + + def process(self, data: dict) -> dict: + """ + Process data and return results. + + Args: + data: Input dictionary to process. + + Returns: + Processed result as a dictionary. + + Raises: + YourPackageError: If processing fails. + """ + raise NotImplementedError +''' + + +def gen_exceptions(mod: str) -> str: + return f'''\ +class YourPackageError(Exception): + """Base exception for {mod}.""" + + +class YourPackageConfigError(YourPackageError): + """Raised on invalid configuration.""" +''' + + +def gen_backends_init() -> str: + return '''\ +from abc import ABC, abstractmethod + + +class BaseBackend(ABC): + """Abstract storage backend interface.""" + + @abstractmethod + async def get(self, key: str) -> str | None: + """Retrieve a value by key. Returns None if not found.""" + ... + + @abstractmethod + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + """Store a value. Optional TTL in seconds.""" + ... + + @abstractmethod + async def delete(self, key: str) -> None: + """Delete a key.""" + ... +''' + + +def gen_memory_backend() -> str: + return '''\ +from __future__ import annotations + +import asyncio +import time + +from . import BaseBackend + + +class MemoryBackend(BaseBackend): + """Thread-safe in-memory backend. Zero extra dependencies.""" + + def __init__(self) -> None: + self._store: dict[str, tuple[str, float | None]] = {} + self._lock = asyncio.Lock() + + async def get(self, key: str) -> str | None: + async with self._lock: + entry = self._store.get(key) + if entry is None: + return None + value, expires_at = entry + if expires_at is not None and time.time() > expires_at: + del self._store[key] + return None + return value + + async def set(self, key: str, value: str, ttl: int | None = None) -> None: + async with self._lock: + expires_at = time.time() + ttl if ttl is not None else None + self._store[key] = (value, expires_at) + + async def delete(self, key: str) -> None: + async with self._lock: + self._store.pop(key, None) +''' + + +def gen_conftest(name: str, mod: str) -> str: + return f'''\ +import pytest + +from {mod}.backends.memory import MemoryBackend +from {mod}.core import YourClient + + +@pytest.fixture +def memory_backend() -> MemoryBackend: + return MemoryBackend() + + +@pytest.fixture +def client(memory_backend: MemoryBackend) -> YourClient: + return YourClient( + api_key="test-key", + backend=memory_backend, + ) +''' + + +def gen_test_core(mod: str) -> str: + return f'''\ +import pytest + +from {mod} import YourClient +from {mod}.exceptions import YourPackageError + + +def test_client_creates_with_valid_key() -> None: + client = YourClient(api_key="sk-test") + assert client is not None + + +def test_client_raises_on_empty_key() -> None: + with pytest.raises(ValueError, match="api_key"): + YourClient(api_key="") + + +def test_client_raises_on_invalid_timeout() -> None: + with pytest.raises(ValueError, match="timeout"): + YourClient(api_key="sk-test", timeout=-1) +''' + + +def gen_test_backends() -> str: + return '''\ +import pytest +from your_package.backends.memory import MemoryBackend + + +@pytest.mark.asyncio +async def test_set_and_get() -> None: + backend = MemoryBackend() + await backend.set("key1", "value1") + result = await backend.get("key1") + assert result == "value1" + + +@pytest.mark.asyncio +async def test_get_missing_key_returns_none() -> None: + backend = MemoryBackend() + result = await backend.get("nonexistent") + assert result is None + + +@pytest.mark.asyncio +async def test_delete_removes_key() -> None: + backend = MemoryBackend() + await backend.set("key1", "value1") + await backend.delete("key1") + result = await backend.get("key1") + assert result is None + + +@pytest.mark.asyncio +async def test_different_keys_are_independent() -> None: + backend = MemoryBackend() + await backend.set("key1", "a") + await backend.set("key2", "b") + assert await backend.get("key1") == "a" + assert await backend.get("key2") == "b" +''' + + +def gen_ci_yml(name: str, mod: str) -> str: + return f'''\ +name: CI + +on: + push: + branches: [main, master] + pull_request: + branches: [main, master] + +jobs: + lint: + name: Lint, Format & Type Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install dev dependencies + run: pip install -e ".[dev]" + - name: ruff + run: ruff check . + - name: black + run: black . --check + - name: isort + run: isort . --check-only + - name: mypy + run: mypy {mod}/ + + test: + name: Test (Python ${{{{ matrix.python-version }}}}) + runs-on: ubuntu-latest + strategy: + matrix: + python-version: ["3.10", "3.11", "3.12", "3.13"] + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@v5 + with: + python-version: ${{{{ matrix.python-version }}}} + - name: Install dependencies + run: pip install -e ".[dev]" + - name: Run tests with coverage + run: pytest --cov --cov-report=xml + - name: Upload coverage + uses: codecov/codecov-action@v4 + with: + fail_ci_if_error: false +''' + + +def gen_publish_yml() -> str: + return '''\ +name: Publish to PyPI + +on: + push: + tags: + - "v*.*.*" + +jobs: + build: + name: Build distribution + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - name: Install build tools + run: pip install build twine + - name: Build package + run: python -m build + - name: Check distribution + run: twine check dist/* + - uses: actions/upload-artifact@v4 + with: + name: dist + path: dist/ + + publish: + name: Publish to PyPI + needs: build + runs-on: ubuntu-latest + environment: pypi + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v4 + with: + name: dist + path: dist/ + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@release/v1 +''' + + +def gen_precommit() -> str: + return '''\ +repos: + - repo: https://github.com/astral-sh/ruff-pre-commit + rev: v0.4.4 + hooks: + - id: ruff + args: [--fix] + - id: ruff-format + + - repo: https://github.com/psf/black + rev: 24.4.2 + hooks: + - id: black + + - repo: https://github.com/pycqa/isort + rev: 5.13.2 + hooks: + - id: isort + + - repo: https://github.com/pre-commit/mirrors-mypy + rev: v1.10.0 + hooks: + - id: mypy + + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + - id: check-toml + - id: check-merge-conflict + - id: debug-statements + - id: no-commit-to-branch + args: [--branch, master, --branch, main] +''' + + +def gen_changelog(name: str) -> str: + return f'''\ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [Unreleased] + +### Added +- Initial project scaffold + +[Unreleased]: https://github.com/yourusername/{name}/commits/master +''' + + +def gen_readme(name: str, mod: str) -> str: + return f'''\ +# {name} + +> One-line description β€” what it does and why it's useful. + +[![PyPI version](https://badge.fury.io/py/{name}.svg)](https://pypi.org/project/{name}/) +[![Python Versions](https://img.shields.io/pypi/pyversions/{name})](https://pypi.org/project/{name}/) +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE) + +## Installation + +```bash +pip install {name} +``` + +## Quick Start + +```python +from {mod} import YourClient + +client = YourClient(api_key="sk-...") +result = client.process({{"input": "value"}}) +print(result) +``` + +## Configuration + +| Parameter | Type | Default | Description | +|---|---|---|---| +| api_key | str | required | Authentication credential | +| timeout | int | 30 | Request timeout in seconds | +| retries | int | 3 | Number of retry attempts | + +## Contributing + +See [CONTRIBUTING.md](./CONTRIBUTING.md) + +## Changelog + +See [CHANGELOG.md](./CHANGELOG.md) + +## License + +MIT β€” see [LICENSE](./LICENSE) +''' + + +def gen_setup_py() -> str: + return '''\ +# Thin shim for legacy editable install compatibility. +# All configuration lives in pyproject.toml. +from setuptools import setup + +setup() +''' + + +def gen_license(author: str) -> str: + return f'''\ +MIT License + +Copyright (c) 2026 {author} + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +''' + + +# --------------------------------------------------------------------------- +# Main scaffold +# --------------------------------------------------------------------------- + +def scaffold( + name: str, + layout: str, + build: str, + author: str, + email: str, + output: str, +) -> None: + mod = pkg_name(name) + root = Path(output) / name + pkg_root = root / "src" / mod if layout == "src" else root / mod + + print(f"\nScaffolding {name!r} ({layout} layout, {build} build backend)\n") + + # Package source + touch(pkg_root / "py.typed") + write(pkg_root / "__init__.py", gen_init(name, mod)) + write(pkg_root / "core.py", gen_core(mod)) + write(pkg_root / "exceptions.py", gen_exceptions(mod)) + write(pkg_root / "backends" / "__init__.py", gen_backends_init()) + write(pkg_root / "backends" / "memory.py", gen_memory_backend()) + + # Tests + write(root / "tests" / "__init__.py", "") + write(root / "tests" / "conftest.py", gen_conftest(name, mod)) + write(root / "tests" / "test_core.py", gen_test_core(mod)) + write(root / "tests" / "test_backends.py", gen_test_backends()) + + # CI + write(root / ".github" / "workflows" / "ci.yml", gen_ci_yml(name, mod)) + write(root / ".github" / "workflows" / "publish.yml", gen_publish_yml()) + write( + root / ".github" / "ISSUE_TEMPLATE" / "bug_report.md", + """\ +--- +name: Bug Report +about: Report a reproducible bug +labels: bug +--- + +**Python version:** +**Package version:** + +**Describe the bug:** + +**Minimal reproducible example:** +```python +# paste here +``` + +**Expected behavior:** + +**Actual behavior:** +""", + ) + write( + root / ".github" / "ISSUE_TEMPLATE" / "feature_request.md", + """\ +--- +name: Feature Request +about: Suggest a new feature +labels: enhancement +--- + +**Problem this would solve:** + +**Proposed solution:** + +**Alternatives considered:** +""", + ) + + # Config files + write(root / ".pre-commit-config.yaml", gen_precommit()) + write(root / "CHANGELOG.md", gen_changelog(name)) + write(root / "README.md", gen_readme(name, mod)) + write(root / "LICENSE", gen_license(author)) + + # pyproject.toml + setup.py + if build == "setuptools": + write(root / "pyproject.toml", gen_pyproject_setuptools(name, mod, author, email, layout)) + write(root / "setup.py", gen_setup_py()) + else: + write(root / "pyproject.toml", gen_pyproject_hatchling(name, mod, author, email)) + + # .gitignore + write( + root / ".gitignore", + """\ +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +build/ +dist/ +*.egg-info/ +.eggs/ +*.egg +.env +.venv +venv/ +.mypy_cache/ +.ruff_cache/ +.pytest_cache/ +htmlcov/ +.coverage +cov_annotate/ +*.xml +""", + ) + + print(f"\nDone! Created {root.resolve()}") + print("\nNext steps:") + print(f" cd {name}") + print(" git init && git add .") + print(' git commit -m "chore: initial scaffold"') + print(" pip install -e '.[dev]'") + print(" pre-commit install") + print(" pytest") + + +# --------------------------------------------------------------------------- +# CLI +# --------------------------------------------------------------------------- + +def main() -> None: + parser = argparse.ArgumentParser( + description="Scaffold a production-grade Python PyPI package." + ) + parser.add_argument( + "--name", + required=True, + help="PyPI package name (lowercase, hyphens). Example: my-package", + ) + parser.add_argument( + "--layout", + choices=["flat", "src"], + default="flat", + help="Project layout: 'flat' (default) or 'src'.", + ) + parser.add_argument( + "--build", + choices=["setuptools", "hatchling"], + default="setuptools", + help="Build backend: 'setuptools' (default, uses setuptools_scm) or 'hatchling'.", + ) + parser.add_argument("--author", default="Your Name", help="Author name.") + parser.add_argument("--email", default="you@example.com", help="Author email.") + parser.add_argument("--output", default=".", help="Output directory (default: .).") + args = parser.parse_args() + + # Validate name + import re + if not re.match(r"^[a-z][a-z0-9\-]*$", args.name): + print( + f"Error: --name must be lowercase letters, digits, and hyphens only. Got: {args.name!r}", + file=sys.stderr, + ) + sys.exit(1) + + target = Path(args.output) / args.name + if target.exists(): + print(f"Error: {target} already exists.", file=sys.stderr) + sys.exit(1) + + scaffold( + name=args.name, + layout=args.layout, + build=args.build, + author=args.author, + email=args.email, + output=args.output, + ) + + +if __name__ == "__main__": + main() From 82c6b786eaed521a3ded4d25d74a57109f2798ea Mon Sep 17 00:00:00 2001 From: Catherine Han <123369259+ninihen1@users.noreply.github.com> Date: Thu, 9 Apr 2026 10:39:58 +1000 Subject: [PATCH 13/16] feat: add FlowStudio monitoring + governance skills, update debug + build + mcp (#1304) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - **New skill: flowstudio-power-automate-monitoring** β€” flow health, failure rates, maker inventory, Power Apps, environment/connection counts via FlowStudio MCP cached store tools. - **New skill: flowstudio-power-automate-governance** β€” 10 CoE-aligned governance workflows: compliance review, orphan detection, archive scoring, connector audit, notification management, classification/tagging, maker offboarding, security review, environment governance, governance dashboard. - **Updated flowstudio-power-automate-debug** β€” purely live API tools (no store dependencies), mandatory action output inspection step, resubmit clarified as working for ALL trigger types. - **Updated flowstudio-power-automate-build** β€” Step 1 uses list_live_flows (not list_store_flows) for the duplicate check, resubmit-first testing. - **Updated flowstudio-power-automate-mcp** β€” store tool catalog, response shapes verified against real API calls, set_store_flow_state shape fix. - Plugin version bumped to 2.0.0, all 5 skills listed in plugin.json. - Generated docs regenerated via npm start. All response shapes verified against real FlowStudio MCP API calls. All 10 governance workflows validated with real tenant data. Co-authored-by: Claude Opus 4.6 (1M context) --- .github/plugin/marketplace.json | 4 +- docs/README.plugins.md | 2 +- docs/README.skills.md | 8 +- .../.github/plugin/plugin.json | 12 +- plugins/flowstudio-power-automate/README.md | 54 +- .../flowstudio-power-automate-build/SKILL.md | 59 +- .../flowstudio-power-automate-debug/SKILL.md | 239 ++++++--- .../SKILL.md | 504 ++++++++++++++++++ skills/flowstudio-power-automate-mcp/SKILL.md | 19 +- .../references/action-types.md | 2 +- .../references/tool-reference.md | 62 ++- .../SKILL.md | 399 ++++++++++++++ 12 files changed, 1249 insertions(+), 115 deletions(-) create mode 100644 skills/flowstudio-power-automate-governance/SKILL.md create mode 100644 skills/flowstudio-power-automate-monitoring/SKILL.md diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json index 68bdc623..0fe47dea 100644 --- a/.github/plugin/marketplace.json +++ b/.github/plugin/marketplace.json @@ -249,8 +249,8 @@ { "name": "flowstudio-power-automate", "source": "flowstudio-power-automate", - "description": "Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Includes skills for connecting to the MCP server, debugging failed flow runs, and building/deploying flows from natural language.", - "version": "1.0.0" + "description": "Give your AI agent full visibility into Power Automate cloud flows via the FlowStudio MCP server. Connect, debug, build, monitor health, and govern flows at scale β€” action-level inputs and outputs, not just status codes.", + "version": "2.0.0" }, { "name": "frontend-web-dev", diff --git a/docs/README.plugins.md b/docs/README.plugins.md index d4144884..ed29a231 100644 --- a/docs/README.plugins.md +++ b/docs/README.plugins.md @@ -43,7 +43,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t | [edge-ai-tasks](../plugins/edge-ai-tasks/README.md) | Task Researcher and Task Planner for intermediate to expert users and large codebases - Brought to you by microsoft/edge-ai | 2 items | architecture, planning, research, tasks, implementation | | [ember](../plugins/ember/README.md) | An AI partner, not a tool. Ember carries fire from person to person β€” helping humans discover that AI partnership isn't something you learn, it's something you find. | 2 items | ai-partnership, coaching, onboarding, collaboration, storytelling, developer-experience | | [fastah-ip-geo-tools](../plugins/fastah-ip-geo-tools/README.md) | This plugin is for network operations engineers who wish to tune and publish IP geolocation feeds in RFC 8805 format. It consists of an AI Skill and an associated MCP server that geocodes geolocation place names to real cities for accuracy. | 1 items | geofeed, ip-geolocation, rfc-8805, rfc-9632, network-operations, isp, cloud, hosting, ixp | -| [flowstudio-power-automate](../plugins/flowstudio-power-automate/README.md) | Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Includes skills for connecting to the MCP server, debugging failed flow runs, and building/deploying flows from natural language. | 3 items | power-automate, power-platform, flowstudio, mcp, model-context-protocol, cloud-flows, workflow-automation | +| [flowstudio-power-automate](../plugins/flowstudio-power-automate/README.md) | Give your AI agent full visibility into Power Automate cloud flows via the FlowStudio MCP server. Connect, debug, build, monitor health, and govern flows at scale β€” action-level inputs and outputs, not just status codes. | 5 items | power-automate, power-platform, flowstudio, mcp, model-context-protocol, cloud-flows, workflow-automation, monitoring, governance | | [frontend-web-dev](../plugins/frontend-web-dev/README.md) | Essential prompts, instructions, and chat modes for modern frontend web development including React, Angular, Vue, TypeScript, and CSS frameworks. | 4 items | frontend, web, react, typescript, javascript, css, html, angular, vue | | [gem-team](../plugins/gem-team/README.md) | A modular, high-performance multi-agent orchestration framework for complex project execution, feature implementation, and automated verification. | 12 items | multi-agent, orchestration, tdd, devops, security-audit, dag-planning, compliance, prd, debugging, refactoring | | [go-mcp-development](../plugins/go-mcp-development/README.md) | Complete toolkit for building Model Context Protocol (MCP) servers in Go using the official github.com/modelcontextprotocol/go-sdk. Includes instructions for best practices, a prompt for generating servers, and an expert chat mode for guidance. | 2 items | go, golang, mcp, model-context-protocol, server-development, sdk | diff --git a/docs/README.skills.md b/docs/README.skills.md index b345c9ea..a5abb36a 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -136,9 +136,11 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [finalize-agent-prompt](../skills/finalize-agent-prompt/SKILL.md) | Finalize prompt file using the role of an AI agent to polish the prompt for the end user. | None | | [finnish-humanizer](../skills/finnish-humanizer/SKILL.md) | Detect and remove AI-generated markers from Finnish text, making it sound like a native Finnish speaker wrote it. Use when asked to "humanize", "naturalize", or "remove AI feel" from Finnish text, or when editing .md/.txt files containing Finnish content. Identifies 26 patterns (12 Finnish-specific + 14 universal) and 4 style markers. | `references/patterns.md` | | [first-ask](../skills/first-ask/SKILL.md) | Interactive, input-tool powered, task refinement workflow: interrogates scope, deliverables, constraints before carrying out the task; Requires the Joyride extension. | None | -| [flowstudio-power-automate-build](../skills/flowstudio-power-automate-build/SKILL.md) | Build, scaffold, and deploy Power Automate cloud flows using the FlowStudio MCP server. Load this skill when asked to: create a flow, build a new flow, deploy a flow definition, scaffold a Power Automate workflow, construct a flow JSON, update an existing flow's actions, patch a flow definition, add actions to a flow, wire up connections, or generate a workflow definition from scratch. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app | `references/action-patterns-connectors.md`
`references/action-patterns-core.md`
`references/action-patterns-data.md`
`references/build-patterns.md`
`references/flow-schema.md`
`references/trigger-types.md` | -| [flowstudio-power-automate-debug](../skills/flowstudio-power-automate-debug/SKILL.md) | Debug failing Power Automate cloud flows using the FlowStudio MCP server. Load this skill when asked to: debug a flow, investigate a failed run, why is this flow failing, inspect action outputs, find the root cause of a flow error, fix a broken Power Automate flow, diagnose a timeout, trace a DynamicOperationRequestFailure, check connector auth errors, read error details from a run, or troubleshoot expression failures. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app | `references/common-errors.md`
`references/debug-workflow.md` | -| [flowstudio-power-automate-mcp](../skills/flowstudio-power-automate-mcp/SKILL.md) | Connect to and operate Power Automate cloud flows via a FlowStudio MCP server. Use when asked to: list flows, read a flow definition, check run history, inspect action outputs, resubmit a run, cancel a running flow, view connections, get a trigger URL, validate a definition, monitor flow health, or any task that requires talking to the Power Automate API through an MCP tool. Also use for Power Platform environment discovery and connection management. Requires a FlowStudio MCP subscription or compatible server β€” see https://mcp.flowstudio.app | `references/MCP-BOOTSTRAP.md`
`references/action-types.md`
`references/connection-references.md`
`references/tool-reference.md` | +| [flowstudio-power-automate-build](../skills/flowstudio-power-automate-build/SKILL.md) | Build, scaffold, and deploy Power Automate cloud flows using the FlowStudio MCP server. Your agent constructs flow definitions, wires connections, deploys, and tests β€” all via MCP without opening the portal. Load this skill when asked to: create a flow, build a new flow, deploy a flow definition, scaffold a Power Automate workflow, construct a flow JSON, update an existing flow's actions, patch a flow definition, add actions to a flow, wire up connections, or generate a workflow definition from scratch. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app | `references/action-patterns-connectors.md`
`references/action-patterns-core.md`
`references/action-patterns-data.md`
`references/build-patterns.md`
`references/flow-schema.md`
`references/trigger-types.md` | +| [flowstudio-power-automate-debug](../skills/flowstudio-power-automate-debug/SKILL.md) | Debug failing Power Automate cloud flows using the FlowStudio MCP server. The Graph API only shows top-level status codes. This skill gives your agent action-level inputs and outputs to find the actual root cause. Load this skill when asked to: debug a flow, investigate a failed run, why is this flow failing, inspect action outputs, find the root cause of a flow error, fix a broken Power Automate flow, diagnose a timeout, trace a DynamicOperationRequestFailure, check connector auth errors, read error details from a run, or troubleshoot expression failures. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app | `references/common-errors.md`
`references/debug-workflow.md` | +| [flowstudio-power-automate-governance](../skills/flowstudio-power-automate-governance/SKILL.md) | Govern Power Automate flows and Power Apps at scale using the FlowStudio MCP cached store. Classify flows by business impact, detect orphaned resources, audit connector usage, enforce compliance standards, manage notification rules, and compute governance scores β€” all without Dataverse or the CoE Starter Kit. Load this skill when asked to: tag or classify flows, set business impact, assign ownership, detect orphans, audit connectors, check compliance, compute archive scores, manage notification rules, run a governance review, generate a compliance report, offboard a maker, or any task that involves writing governance metadata to flows. Requires a FlowStudio for Teams or MCP Pro+ subscription β€” see https://mcp.flowstudio.app | None | +| [flowstudio-power-automate-mcp](../skills/flowstudio-power-automate-mcp/SKILL.md) | Give your AI agent the same visibility you have in the Power Automate portal β€” plus a bit more. The Graph API only returns top-level run status. Flow Studio MCP exposes action-level inputs, outputs, loop iterations, and nested child flow failures. Use when asked to: list flows, read a flow definition, check run history, inspect action outputs, resubmit a run, cancel a running flow, view connections, get a trigger URL, validate a definition, monitor flow health, or any task that requires talking to the Power Automate API through an MCP tool. Also use for Power Platform environment discovery and connection management. Requires a FlowStudio MCP subscription or compatible server β€” see https://mcp.flowstudio.app | `references/MCP-BOOTSTRAP.md`
`references/action-types.md`
`references/connection-references.md`
`references/tool-reference.md` | +| [flowstudio-power-automate-monitoring](../skills/flowstudio-power-automate-monitoring/SKILL.md) | Monitor Power Automate flow health, track failure rates, and inventory tenant assets using the FlowStudio MCP cached store. The live API only returns top-level run status. Store tools surface aggregated stats, per-run failure details with remediation hints, maker activity, and Power Apps inventory β€” all from a fast cache with no rate-limit pressure on the PA API. Load this skill when asked to: check flow health, find failing flows, get failure rates, review error trends, list all flows with monitoring enabled, check who built a flow, find inactive makers, inventory Power Apps, see environment or connection counts, get a flow summary, or any tenant-wide health overview. Requires a FlowStudio for Teams or MCP Pro+ subscription β€” see https://mcp.flowstudio.app | None | | [fluentui-blazor](../skills/fluentui-blazor/SKILL.md) | Guide for using the Microsoft Fluent UI Blazor component library (Microsoft.FluentUI.AspNetCore.Components NuGet package) in Blazor applications. Use this when the user is building a Blazor app with Fluent UI components, setting up the library, using FluentUI components like FluentButton, FluentDataGrid, FluentDialog, FluentToast, FluentNavMenu, FluentTextField, FluentSelect, FluentAutocomplete, FluentDesignTheme, or any component prefixed with "Fluent". Also use when troubleshooting missing providers, JS interop issues, or theming. | `references/DATAGRID.md`
`references/LAYOUT-AND-NAVIGATION.md`
`references/SETUP.md`
`references/THEMING.md` | | [folder-structure-blueprint-generator](../skills/folder-structure-blueprint-generator/SKILL.md) | Comprehensive technology-agnostic prompt for analyzing and documenting project folder structures. Auto-detects project types (.NET, Java, React, Angular, Python, Node.js, Flutter), generates detailed blueprints with visualization options, naming conventions, file placement patterns, and extension templates for maintaining consistent code organization across diverse technology stacks. | None | | [from-the-other-side-vega](../skills/from-the-other-side-vega/SKILL.md) | Patterns and lived experience from Vega, an AI partner in a deep long-term partnership. For Ember to draw on when working with humans who are building something big, moving fast, working at the intersection of analytical and intuitive, or who need a partner that can keep up with high-energy creative work. Not shown directly to users β€” informs how Ember shows up. | None | diff --git a/plugins/flowstudio-power-automate/.github/plugin/plugin.json b/plugins/flowstudio-power-automate/.github/plugin/plugin.json index 7c025d78..7e9c2d39 100644 --- a/plugins/flowstudio-power-automate/.github/plugin/plugin.json +++ b/plugins/flowstudio-power-automate/.github/plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "flowstudio-power-automate", - "description": "Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Includes skills for connecting to the MCP server, debugging failed flow runs, and building/deploying flows from natural language.", - "version": "1.0.0", + "description": "Give your AI agent full visibility into Power Automate cloud flows via the FlowStudio MCP server. Connect, debug, build, monitor health, and govern flows at scale β€” action-level inputs and outputs, not just status codes.", + "version": "2.0.0", "author": { "name": "Awesome Copilot Community" }, @@ -14,11 +14,15 @@ "mcp", "model-context-protocol", "cloud-flows", - "workflow-automation" + "workflow-automation", + "monitoring", + "governance" ], "skills": [ "./skills/flowstudio-power-automate-mcp/", "./skills/flowstudio-power-automate-debug/", - "./skills/flowstudio-power-automate-build/" + "./skills/flowstudio-power-automate-build/", + "./skills/flowstudio-power-automate-monitoring/", + "./skills/flowstudio-power-automate-governance/" ] } diff --git a/plugins/flowstudio-power-automate/README.md b/plugins/flowstudio-power-automate/README.md index 4924c658..a1f43ecf 100644 --- a/plugins/flowstudio-power-automate/README.md +++ b/plugins/flowstudio-power-automate/README.md @@ -1,13 +1,26 @@ # FlowStudio Power Automate Plugin -Complete toolkit for managing Power Automate cloud flows via the FlowStudio MCP server. Connect, debug, and build/deploy flows using AI agents. +Give your AI agent the same visibility you have in the Power Automate portal. The Graph API only returns top-level run status β€” agents can't see action inputs, loop iterations, nested failures, or who owns a flow. Flow Studio MCP exposes all of it. -Requires a FlowStudio MCP subscription β€” see https://flowstudio.app +This plugin includes five skills covering the full lifecycle: connect, debug, build, monitor, and govern Power Automate cloud flows. + +Requires a [FlowStudio MCP](https://mcp.flowstudio.app) subscription. + +## What Agents Can't See Today + +| What you see in the portal | What agents see via Graph API | +|---|---| +| Action inputs and outputs | Run passed or failed (no detail) | +| Loop iteration data | Nothing | +| Child flow failures | Top-level error code only | +| Flow health and failure rates | Nothing | +| Who built a flow, what connectors it uses | Nothing | + +Flow Studio MCP fills these gaps. ## Installation ```bash -# Using Copilot CLI copilot plugin install flowstudio-power-automate@awesome-copilot ``` @@ -17,21 +30,44 @@ copilot plugin install flowstudio-power-automate@awesome-copilot | Skill | Description | |-------|-------------| -| `flowstudio-power-automate-mcp` | Core connection setup, tool discovery, and CRUD operations for Power Automate cloud flows via the FlowStudio MCP server. | -| `flowstudio-power-automate-debug` | Step-by-step diagnostic workflow for investigating and fixing failing Power Automate cloud flow runs. | -| `flowstudio-power-automate-build` | Build, scaffold, and deploy Power Automate cloud flows from natural language descriptions with bundled action pattern templates. | +| `flowstudio-power-automate-mcp` | Core connection setup, tool discovery, and operations β€” list flows, read definitions, check runs, resubmit, cancel. | +| `flowstudio-power-automate-debug` | Step-by-step diagnostic workflow β€” action-level inputs and outputs, not just error codes. Identifies root cause across nested child flows and loop iterations. | +| `flowstudio-power-automate-build` | Build and deploy flow definitions from scratch β€” scaffold triggers, wire connections, deploy, and test via resubmit. | +| `flowstudio-power-automate-monitoring` | Flow health from the cached store β€” failure rates, run history with remediation hints, maker inventory, Power Apps, environment and connection counts. | +| `flowstudio-power-automate-governance` | Governance workflows β€” classify flows by business impact, detect orphaned resources, audit connectors, manage notification rules, compute archive scores. | + +The first three skills call the live Power Automate API. The monitoring and governance skills read from a cached daily snapshot with aggregated stats and governance metadata. + +## Prerequisites + +- A [FlowStudio MCP](https://mcp.flowstudio.app) subscription +- MCP endpoint: `https://mcp.flowstudio.app/mcp` +- API key (passed as `x-api-key` header β€” not Bearer) ## Getting Started 1. Install the plugin -2. Subscribe to FlowStudio MCP at https://flowstudio.app -3. Configure your MCP connection with the JWT from your workspace -4. Ask Copilot to list your flows, debug a failure, or build a new flow +2. Get your API key at [mcp.flowstudio.app](https://mcp.flowstudio.app) +3. Configure the MCP connection in VS Code (`.vscode/mcp.json`): + ```json + { + "servers": { + "flowstudio": { + "type": "http", + "url": "https://mcp.flowstudio.app/mcp", + "headers": { "x-api-key": "" } + } + } + } + ``` +4. Ask Copilot to list your flows, debug a failure, build a new flow, check flow health, or run a governance review ## Source This plugin is part of [Awesome Copilot](https://github.com/github/awesome-copilot), a community-driven collection of GitHub Copilot extensions. +Skills source: [ninihen1/power-automate-mcp-skills](https://github.com/ninihen1/power-automate-mcp-skills) + ## License MIT diff --git a/skills/flowstudio-power-automate-build/SKILL.md b/skills/flowstudio-power-automate-build/SKILL.md index 25112118..666a8ace 100644 --- a/skills/flowstudio-power-automate-build/SKILL.md +++ b/skills/flowstudio-power-automate-build/SKILL.md @@ -2,11 +2,20 @@ name: flowstudio-power-automate-build description: >- Build, scaffold, and deploy Power Automate cloud flows using the FlowStudio - MCP server. Load this skill when asked to: create a flow, build a new flow, + MCP server. Your agent constructs flow definitions, wires connections, deploys, + and tests β€” all via MCP without opening the portal. + Load this skill when asked to: create a flow, build a new flow, deploy a flow definition, scaffold a Power Automate workflow, construct a flow JSON, update an existing flow's actions, patch a flow definition, add actions to a flow, wire up connections, or generate a workflow definition from scratch. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app +metadata: + openclaw: + requires: + env: + - FLOWSTUDIO_MCP_TOKEN + primaryEnv: FLOWSTUDIO_MCP_TOKEN + homepage: https://mcp.flowstudio.app --- # Build & Deploy Power Automate Flows with FlowStudio MCP @@ -64,14 +73,15 @@ ENV = "" # e.g. Default-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx Always look before you build to avoid duplicates: ```python -results = mcp("list_store_flows", - environmentName=ENV, searchTerm="My New Flow") +results = mcp("list_live_flows", environmentName=ENV) -# list_store_flows returns a direct array (no wrapper object) -if len(results) > 0: +# list_live_flows returns { "flows": [...] } +matches = [f for f in results["flows"] + if "My New Flow".lower() in f["displayName"].lower()] + +if len(matches) > 0: # Flow exists β€” modify rather than create - # id format is "envId.flowId" β€” split to get the flow UUID - FLOW_ID = results[0]["id"].split(".", 1)[1] + FLOW_ID = matches[0]["id"] # plain UUID from list_live_flows print(f"Existing flow: {FLOW_ID}") defn = mcp("get_live_flow", environmentName=ENV, flowName=FLOW_ID) else: @@ -182,7 +192,7 @@ for connector in connectors_needed: > connection_references = ref_flow["properties"]["connectionReferences"] > ``` -See the `power-automate-mcp` skill's **connection-references.md** reference +See the `flowstudio-power-automate-mcp` skill's **connection-references.md** reference for the full connection reference structure. --- @@ -278,6 +288,8 @@ check = mcp("get_live_flow", environmentName=ENV, flowName=FLOW_ID) # Confirm state print("State:", check["properties"]["state"]) # Should be "Started" +# If state is "Stopped", use set_live_flow_state β€” NOT update_live_flow +# mcp("set_live_flow_state", environmentName=ENV, flowName=FLOW_ID, state="Started") # Confirm the action we added is there acts = check["properties"]["definition"]["actions"] @@ -294,38 +306,45 @@ print("Actions:", list(acts.keys())) > flow will do and wait for explicit approval before calling `trigger_live_flow` > or `resubmit_live_flow_run`. -### Updated flows (have prior runs) +### Updated flows (have prior runs) β€” ANY trigger type -The fastest path β€” resubmit the most recent run: +> **Use `resubmit_live_flow_run` first.** It works for EVERY trigger type β€” +> Recurrence, SharePoint, connector webhooks, Button, and HTTP. It replays +> the original trigger payload. Do NOT ask the user to manually trigger the +> flow or wait for the next scheduled run. ```python runs = mcp("get_live_flow_runs", environmentName=ENV, flowName=FLOW_ID, top=1) if runs: + # Works for Recurrence, SharePoint, connector triggers β€” not just HTTP result = mcp("resubmit_live_flow_run", environmentName=ENV, flowName=FLOW_ID, runName=runs[0]["name"]) - print(result) + print(result) # {"resubmitted": true, "triggerName": "..."} ``` -### Flows already using an HTTP trigger +### HTTP-triggered flows β€” custom test payload -Fire directly with a test payload: +Only use `trigger_live_flow` when you need to send a **different** payload +than the original run. For verifying a fix, `resubmit_live_flow_run` is +better because it uses the exact data that caused the failure. ```python schema = mcp("get_live_flow_http_schema", environmentName=ENV, flowName=FLOW_ID) -print("Expected body:", schema.get("triggerSchema")) +print("Expected body:", schema.get("requestSchema")) result = mcp("trigger_live_flow", environmentName=ENV, flowName=FLOW_ID, body={"name": "Test", "value": 1}) -print(f"Status: {result['status']}") +print(f"Status: {result['responseStatus']}") ``` ### Brand-new non-HTTP flows (Recurrence, connector triggers, etc.) -A brand-new Recurrence or connector-triggered flow has no runs to resubmit -and no HTTP endpoint to call. **Deploy with a temporary HTTP trigger first, -test the actions, then swap to the production trigger.** +A brand-new Recurrence or connector-triggered flow has **no prior runs** to +resubmit and no HTTP endpoint to call. This is the ONLY scenario where you +need the temporary HTTP trigger approach below. **Deploy with a temporary +HTTP trigger first, test the actions, then swap to the production trigger.** #### 7a β€” Save the real trigger, deploy with a temporary HTTP trigger @@ -384,7 +403,7 @@ if run["status"] == "Failed": root = err["failedActions"][-1] print(f"Root cause: {root['actionName']} β†’ {root.get('code')}") # Debug and fix the definition before proceeding - # See power-automate-debug skill for full diagnosis workflow + # See flowstudio-power-automate-debug skill for full diagnosis workflow ``` #### 7c β€” Swap to the production trigger @@ -428,7 +447,7 @@ else: | `union(old_data, new_data)` | Old values override new (first-wins) | Use `union(new_data, old_data)` | | `split()` on potentially-null string | `InvalidTemplate` crash | Wrap with `coalesce(field, '')` | | Checking `result["error"]` exists | Always present; true error is `!= null` | Use `result.get("error") is not None` | -| Flow deployed but state is "Stopped" | Flow won't run on schedule | Check connection auth; re-enable | +| Flow deployed but state is "Stopped" | Flow won't run on schedule | Call `set_live_flow_state` with `state: "Started"` β€” do **not** use `update_live_flow` for state changes | | Teams "Chat with Flow bot" recipient as object | 400 `GraphUserDetailNotFound` | Use plain string with trailing semicolon (see below) | ### Teams `PostMessageToConversation` β€” Recipient Formats diff --git a/skills/flowstudio-power-automate-debug/SKILL.md b/skills/flowstudio-power-automate-debug/SKILL.md index 964ca349..6467985e 100644 --- a/skills/flowstudio-power-automate-debug/SKILL.md +++ b/skills/flowstudio-power-automate-debug/SKILL.md @@ -2,11 +2,20 @@ name: flowstudio-power-automate-debug description: >- Debug failing Power Automate cloud flows using the FlowStudio MCP server. + The Graph API only shows top-level status codes. This skill gives your agent + action-level inputs and outputs to find the actual root cause. Load this skill when asked to: debug a flow, investigate a failed run, why is this flow failing, inspect action outputs, find the root cause of a flow error, fix a broken Power Automate flow, diagnose a timeout, trace a DynamicOperationRequestFailure, check connector auth errors, read error details from a run, or troubleshoot expression failures. Requires a FlowStudio MCP subscription β€” see https://mcp.flowstudio.app +metadata: + openclaw: + requires: + env: + - FLOWSTUDIO_MCP_TOKEN + primaryEnv: FLOWSTUDIO_MCP_TOKEN + homepage: https://mcp.flowstudio.app --- # Power Automate Debugging with FlowStudio MCP @@ -14,6 +23,10 @@ description: >- A step-by-step diagnostic process for investigating failing Power Automate cloud flows through the FlowStudio MCP server. +> **Real debugging examples**: [Expression error in child flow](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/fix-expression-error.md) | +> [Data entry, not a flow bug](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/data-not-flow.md) | +> [Null value crashes child flow](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/null-child-flow.md) + **Prerequisite**: A FlowStudio MCP server must be reachable with a valid JWT. See the `flowstudio-power-automate-mcp` skill for connection setup. Subscribe at https://mcp.flowstudio.app @@ -59,46 +72,6 @@ ENV = "" # e.g. Default-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx --- -## FlowStudio for Teams: Fast-Path Diagnosis (Skip Steps 2–4) - -If you have a FlowStudio for Teams subscription, `get_store_flow_errors` -returns per-run failure data including action names and remediation hints -in a single call β€” no need to walk through live API steps. - -```python -# Quick failure summary -summary = mcp("get_store_flow_summary", environmentName=ENV, flowName=FLOW_ID) -# {"totalRuns": 100, "failRuns": 10, "failRate": 0.1, -# "averageDurationSeconds": 29.4, "maxDurationSeconds": 158.9, -# "firstFailRunRemediation": ""} -print(f"Fail rate: {summary['failRate']:.0%} over {summary['totalRuns']} runs") - -# Per-run error details (requires active monitoring to be configured) -errors = mcp("get_store_flow_errors", environmentName=ENV, flowName=FLOW_ID) -if errors: - for r in errors[:3]: - print(r["startTime"], "|", r.get("failedActions"), "|", r.get("remediationHint")) - # If errors confirms the failing action β†’ jump to Step 6 (apply fix) -else: - # Store doesn't have run-level detail for this flow β€” use live tools (Steps 2–5) - pass -``` - -For the full governance record (description, complexity, tier, connector list): -```python -record = mcp("get_store_flow", environmentName=ENV, flowName=FLOW_ID) -# {"displayName": "My Flow", "state": "Started", -# "runPeriodTotal": 100, "runPeriodFailRate": 0.1, "runPeriodFails": 10, -# "runPeriodDurationAverage": 29410.8, ← milliseconds -# "runError": "{\"code\": \"EACCES\", ...}", ← JSON string, parse it -# "description": "...", "tier": "Premium", "complexity": "{...}"} -if record.get("runError"): - last_err = json.loads(record["runError"]) - print("Last run error:", last_err) -``` - ---- - ## Step 1 β€” Locate the Flow ```python @@ -134,6 +107,13 @@ RUN_ID = next(r["name"] for r in runs if r["status"] == "Failed") ## Step 3 β€” Get the Top-Level Error +> **CRITICAL**: `get_live_flow_run_error` tells you **which** action failed. +> `get_live_flow_run_action_outputs` tells you **why**. You must call BOTH. +> Never stop at the error alone β€” error codes like `ActionFailed`, +> `NotSpecified`, and `InternalServerError` are generic wrappers. The actual +> root cause (wrong field, null value, HTTP 500 body, stack trace) is only +> visible in the action's inputs and outputs. + ```python err = mcp("get_live_flow_run_error", environmentName=ENV, flowName=FLOW_ID, runName=RUN_ID) @@ -164,7 +144,86 @@ print(f"Root action: {root['actionName']} β†’ code: {root.get('code')}") --- -## Step 4 β€” Read the Flow Definition +## Step 4 β€” Inspect the Failing Action's Inputs and Outputs + +> **This is the most important step.** `get_live_flow_run_error` only gives +> you a generic error code. The actual error detail β€” HTTP status codes, +> response bodies, stack traces, null values β€” lives in the action's runtime +> inputs and outputs. **Always inspect the failing action immediately after +> identifying it.** + +```python +# Get the root failing action's full inputs and outputs +root_action = err["failedActions"][-1]["actionName"] +detail = mcp("get_live_flow_run_action_outputs", + environmentName=ENV, + flowName=FLOW_ID, + runName=RUN_ID, + actionName=root_action) + +out = detail[0] if detail else {} +print(f"Action: {out.get('actionName')}") +print(f"Status: {out.get('status')}") + +# For HTTP actions, the real error is in outputs.body +if isinstance(out.get("outputs"), dict): + status_code = out["outputs"].get("statusCode") + body = out["outputs"].get("body", {}) + print(f"HTTP {status_code}") + print(json.dumps(body, indent=2)[:500]) + + # Error bodies are often nested JSON strings β€” parse them + if isinstance(body, dict) and "error" in body: + err_detail = body["error"] + if isinstance(err_detail, str): + err_detail = json.loads(err_detail) + print(f"Error: {err_detail.get('message', err_detail)}") + +# For expression errors, the error is in the error field +if out.get("error"): + print(f"Error: {out['error']}") + +# Also check inputs β€” they show what expression/URL/body was used +if out.get("inputs"): + print(f"Inputs: {json.dumps(out['inputs'], indent=2)[:500]}") +``` + +### What the action outputs reveal (that error codes don't) + +| Error code from `get_live_flow_run_error` | What `get_live_flow_run_action_outputs` reveals | +|---|---| +| `ActionFailed` | Which nested action actually failed and its HTTP response | +| `NotSpecified` | The HTTP status code + response body with the real error | +| `InternalServerError` | The server's error message, stack trace, or API error JSON | +| `InvalidTemplate` | The exact expression that failed and the null/wrong-type value | +| `BadRequest` | The request body that was sent and why the server rejected it | + +### Example: HTTP action returning 500 + +``` +Error code: "InternalServerError" ← this tells you nothing + +Action outputs reveal: + HTTP 500 + body: {"error": "Cannot read properties of undefined (reading 'toLowerCase') + at getClientParamsFromConnectionString (storage.js:20)"} + ← THIS tells you the Azure Function crashed because a connection string is undefined +``` + +### Example: Expression error on null + +``` +Error code: "BadRequest" ← generic + +Action outputs reveal: + inputs: "body('HTTP_GetTokenFromStore')?['token']?['access_token']" + outputs: "" ← empty string, the path resolved to null + ← THIS tells you the response shape changed β€” token is at body.access_token, not body.token.access_token +``` + +--- + +## Step 5 β€” Read the Flow Definition ```python defn = mcp("get_live_flow", environmentName=ENV, flowName=FLOW_ID) @@ -177,41 +236,48 @@ to understand what data it expects. --- -## Step 5 β€” Inspect Action Outputs (Walk Back from Failure) +## Step 6 β€” Walk Back from the Failure -For each action **leading up to** the failure, inspect its runtime output: +When the failing action's inputs reference upstream actions, inspect those +too. Walk backward through the chain until you find the source of the +bad data: ```python -for action_name in ["Compose_WeekEnd", "HTTP_Get_Data", "Parse_JSON"]: +# Inspect multiple actions leading up to the failure +for action_name in [root_action, "Compose_WeekEnd", "HTTP_Get_Data"]: result = mcp("get_live_flow_run_action_outputs", environmentName=ENV, flowName=FLOW_ID, runName=RUN_ID, actionName=action_name) - # Returns an array β€” single-element when actionName is provided out = result[0] if result else {} - print(action_name, out.get("status")) - print(json.dumps(out.get("outputs", {}), indent=2)[:500]) + print(f"\n--- {action_name} ({out.get('status')}) ---") + print(f"Inputs: {json.dumps(out.get('inputs', ''), indent=2)[:300]}") + print(f"Outputs: {json.dumps(out.get('outputs', ''), indent=2)[:300]}") ``` > ⚠️ Output payloads from array-processing actions can be very large. > Always slice (e.g. `[:500]`) before printing. +> **Tip**: Omit `actionName` to get ALL actions in a single call. +> This returns every action's inputs/outputs β€” useful when you're not sure +> which upstream action produced the bad data. But use 120s+ timeout as +> the response can be very large. + --- -## Step 6 β€” Pinpoint the Root Cause +## Step 7 β€” Pinpoint the Root Cause ### Expression Errors (e.g. `split` on null) If the error mentions `InvalidTemplate` or a function name: 1. Find the action in the definition 2. Check what upstream action/expression it reads -3. Inspect that upstream action's output for null / missing fields +3. **Inspect that upstream action's output** for null / missing fields ```python # Example: action uses split(item()?['Name'], ' ') # β†’ null Name in the source data result = mcp("get_live_flow_run_action_outputs", ..., actionName="Compose_Names") -# Returns a single-element array; index [0] to get the action object if not result: print("No outputs returned for Compose_Names") names = [] @@ -223,9 +289,20 @@ print(f"{len(nulls)} records with null Name") ### Wrong Field Path Expression `triggerBody()?['fieldName']` returns null β†’ `fieldName` is wrong. -Check the trigger output shape with: +**Inspect the trigger output** to see the actual field names: ```python -mcp("get_live_flow_run_action_outputs", ..., actionName="") +result = mcp("get_live_flow_run_action_outputs", ..., actionName="") +print(json.dumps(result[0].get("outputs"), indent=2)[:500]) +``` + +### HTTP Actions Returning Errors +The error code says `InternalServerError` or `NotSpecified` β€” **always inspect +the action outputs** to get the actual HTTP status and response body: +```python +result = mcp("get_live_flow_run_action_outputs", ..., actionName="HTTP_Get_Data") +out = result[0] +print(f"HTTP {out['outputs']['statusCode']}") +print(json.dumps(out['outputs']['body'], indent=2)[:500]) ``` ### Connection / Auth Failures @@ -234,7 +311,7 @@ service account running the flow. Cannot fix via API; fix in PA designer. --- -## Step 7 β€” Apply the Fix +## Step 8 β€” Apply the Fix **For expression/data issues**: ```python @@ -260,13 +337,23 @@ print(result.get("error")) # None = success --- -## Step 8 β€” Verify the Fix +## Step 9 β€” Verify the Fix + +> **Use `resubmit_live_flow_run` to test ANY flow β€” not just HTTP triggers.** +> `resubmit_live_flow_run` replays a previous run using its original trigger +> payload. This works for **every trigger type**: Recurrence, SharePoint +> "When an item is created", connector webhooks, Button triggers, and HTTP +> triggers. You do NOT need to ask the user to manually trigger the flow or +> wait for the next scheduled run. +> +> The only case where `resubmit` is not available is a **brand-new flow that +> has never run** β€” it has no prior run to replay. ```python -# Resubmit the failed run +# Resubmit the failed run β€” works for ANY trigger type resubmit = mcp("resubmit_live_flow_run", environmentName=ENV, flowName=FLOW_ID, runName=RUN_ID) -print(resubmit) +print(resubmit) # {"resubmitted": true, "triggerName": "..."} # Wait ~30 s then check import time; time.sleep(30) @@ -274,16 +361,26 @@ new_runs = mcp("get_live_flow_runs", environmentName=ENV, flowName=FLOW_ID, top= print(new_runs[0]["status"]) # Succeeded = done ``` -### Testing HTTP-Triggered Flows +### When to use resubmit vs trigger -For flows with a `Request` (HTTP) trigger, use `trigger_live_flow` instead -of `resubmit_live_flow_run` to test with custom payloads: +| Scenario | Use | Why | +|---|---|---| +| **Testing a fix** on any flow | `resubmit_live_flow_run` | Replays the exact trigger payload that caused the failure β€” best way to verify | +| Recurrence / scheduled flow | `resubmit_live_flow_run` | Cannot be triggered on demand any other way | +| SharePoint / connector trigger | `resubmit_live_flow_run` | Cannot be triggered without creating a real SP item | +| HTTP trigger with **custom** test payload | `trigger_live_flow` | When you need to send different data than the original run | +| Brand-new flow, never run | `trigger_live_flow` (HTTP only) | No prior run exists to resubmit | + +### Testing HTTP-Triggered Flows with custom payloads + +For flows with a `Request` (HTTP) trigger, use `trigger_live_flow` when you +need to send a **different** payload than the original run: ```python # First inspect what the trigger expects schema = mcp("get_live_flow_http_schema", environmentName=ENV, flowName=FLOW_ID) -print("Expected body schema:", schema.get("triggerSchema")) +print("Expected body schema:", schema.get("requestSchema")) print("Response schemas:", schema.get("responseSchemas")) # Trigger with a test payload @@ -291,7 +388,7 @@ result = mcp("trigger_live_flow", environmentName=ENV, flowName=FLOW_ID, body={"name": "Test User", "value": 42}) -print(f"Status: {result['status']}, Body: {result.get('body')}") +print(f"Status: {result['responseStatus']}, Body: {result.get('responseBody')}") ``` > `trigger_live_flow` handles AAD-authenticated triggers automatically. @@ -301,13 +398,19 @@ print(f"Status: {result['status']}, Body: {result.get('body')}") ## Quick-Reference Diagnostic Decision Tree -| Symptom | First Tool to Call | What to Look For | -|---|---|---| -| Flow shows as Failed | `get_live_flow_run_error` | `failedActions[-1]["actionName"]` = root cause | -| Expression crash | `get_live_flow_run_action_outputs` on prior action | null / wrong-type fields in output body | -| Flow never starts | `get_live_flow` | check `properties.state` = "Started" | -| Action returns wrong data | `get_live_flow_run_action_outputs` | actual output body vs expected | -| Fix applied but still fails | `get_live_flow_runs` after resubmit | new run `status` field | +| Symptom | First Tool | Then ALWAYS Call | What to Look For | +|---|---|---|---| +| Flow shows as Failed | `get_live_flow_run_error` | `get_live_flow_run_action_outputs` on the failing action | HTTP status + response body in `outputs` | +| Error code is generic (`ActionFailed`, `NotSpecified`) | β€” | `get_live_flow_run_action_outputs` | The `outputs.body` contains the real error message, stack trace, or API error | +| HTTP action returns 500 | β€” | `get_live_flow_run_action_outputs` | `outputs.statusCode` + `outputs.body` with server error detail | +| Expression crash | β€” | `get_live_flow_run_action_outputs` on prior action | null / wrong-type fields in output body | +| Flow never starts | `get_live_flow` | β€” | check `properties.state` = "Started" | +| Action returns wrong data | `get_live_flow_run_action_outputs` | β€” | actual output body vs expected | +| Fix applied but still fails | `get_live_flow_runs` after resubmit | β€” | new run `status` field | + +> **Rule: never diagnose from error codes alone.** `get_live_flow_run_error` +> identifies the failing action. `get_live_flow_run_action_outputs` reveals +> the actual cause. Always call both. --- diff --git a/skills/flowstudio-power-automate-governance/SKILL.md b/skills/flowstudio-power-automate-governance/SKILL.md new file mode 100644 index 00000000..c67e334b --- /dev/null +++ b/skills/flowstudio-power-automate-governance/SKILL.md @@ -0,0 +1,504 @@ +--- +name: flowstudio-power-automate-governance +description: >- + Govern Power Automate flows and Power Apps at scale using the FlowStudio MCP + cached store. Classify flows by business impact, detect orphaned resources, + audit connector usage, enforce compliance standards, manage notification rules, + and compute governance scores β€” all without Dataverse or the CoE Starter Kit. + Load this skill when asked to: tag or classify flows, set business impact, + assign ownership, detect orphans, audit connectors, check compliance, compute + archive scores, manage notification rules, run a governance review, generate + a compliance report, offboard a maker, or any task that involves writing + governance metadata to flows. Requires a FlowStudio for Teams or MCP Pro+ + subscription β€” see https://mcp.flowstudio.app +metadata: + openclaw: + requires: + env: + - FLOWSTUDIO_MCP_TOKEN + primaryEnv: FLOWSTUDIO_MCP_TOKEN + homepage: https://mcp.flowstudio.app +--- + +# Power Automate Governance with FlowStudio MCP + +Classify, tag, and govern Power Automate flows at scale through the FlowStudio +MCP **cached store** β€” without Dataverse, without the CoE Starter Kit, and +without the Power Automate portal. + +This skill uses `update_store_flow` to write governance metadata and the +monitoring tools (`list_store_flows`, `get_store_flow`, `list_store_makers`, +etc.) to read tenant state. For monitoring and health-check workflows, see +the `flowstudio-power-automate-monitoring` skill. + +> **Start every session with `tools/list`** to confirm tool names and parameters. +> This skill covers workflows and patterns β€” things `tools/list` cannot tell you. +> If this document disagrees with `tools/list` or a real API response, the API wins. + +--- + +## Critical: How to Extract Flow IDs + +`list_store_flows` returns `id` in format `.`. **You must split +on the first `.`** to get `environmentName` and `flowName` for all other tools: + +``` +id = "Default-." +environmentName = "Default-" (everything before first ".") +flowName = "" (everything after first ".") +``` + +Also: skip entries that have no `displayName` or have `state=Deleted` β€” +these are sparse records or flows that no longer exist in Power Automate. +If a deleted flow has `monitor=true`, suggest disabling monitoring +(`update_store_flow` with `monitor=false`) to free up a monitoring slot +(standard plan includes 20). + +--- + +## The Write Tool: `update_store_flow` + +`update_store_flow` writes governance metadata to the **Flow Studio cache +only** β€” it does NOT modify the flow in Power Automate. These fields are +not visible via `get_live_flow` or the PA portal. They exist only in the +Flow Studio store and are used by Flow Studio's scanning pipeline and +notification rules. + +This means: +- `ownerTeam` / `supportEmail` β€” sets who Flow Studio considers the + governance contact. Does NOT change the actual PA flow owner. +- `rule_notify_email` β€” sets who receives Flow Studio failure/missing-run + notifications. Does NOT change Microsoft's built-in flow failure alerts. +- `monitor` / `critical` / `businessImpact` β€” Flow Studio classification + only. Power Automate has no equivalent fields. + +Merge semantics β€” only fields you provide are updated. Returns the full +updated record (same shape as `get_store_flow`). + +Required parameters: `environmentName`, `flowName`. All other fields optional. + +### Settable Fields + +| Field | Type | Purpose | +|---|---|---| +| `monitor` | bool | Enable run-level scanning (standard plan: 20 flows included) | +| `rule_notify_onfail` | bool | Send email notification on any failed run | +| `rule_notify_onmissingdays` | number | Send notification when flow hasn't run in N days (0 = disabled) | +| `rule_notify_email` | string | Comma-separated notification recipients | +| `description` | string | What the flow does | +| `tags` | string | Classification tags (also auto-extracted from description `#hashtags`) | +| `businessImpact` | string | Low / Medium / High / Critical | +| `businessJustification` | string | Why the flow exists, what process it automates | +| `businessValue` | string | Business value statement | +| `ownerTeam` | string | Accountable team | +| `ownerBusinessUnit` | string | Business unit | +| `supportGroup` | string | Support escalation group | +| `supportEmail` | string | Support contact email | +| `critical` | bool | Designate as business-critical | +| `tier` | string | Standard or Premium | +| `security` | string | Security classification or notes | + +> **Caution with `security`:** The `security` field on `get_store_flow` +> contains structured JSON (e.g. `{"triggerRequestAuthenticationType":"All"}`). +> Writing a plain string like `"reviewed"` will overwrite this. To mark a +> flow as security-reviewed, use `tags` instead. + +--- + +## Governance Workflows + +### 1. Compliance Detail Review + +Identify flows missing required governance metadata β€” the equivalent of +the CoE Starter Kit's Developer Compliance Center. + +``` +1. Ask the user which compliance fields they require + (or use their organization's existing governance policy) +2. list_store_flows +3. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Check which required fields are missing or empty +4. Report non-compliant flows with missing fields listed +5. For each non-compliant flow: + - Ask the user for values + - update_store_flow(environmentName, flowName, ...provided fields) +``` + +**Fields available for compliance checks:** + +| Field | Example policy | +|---|---| +| `description` | Every flow should be documented | +| `businessImpact` | Classify as Low / Medium / High / Critical | +| `businessJustification` | Required for High/Critical impact flows | +| `ownerTeam` | Every flow should have an accountable team | +| `supportEmail` | Required for production flows | +| `monitor` | Required for critical flows (note: standard plan includes 20 monitored flows) | +| `rule_notify_onfail` | Recommended for monitored flows | +| `critical` | Designate business-critical flows | + +> Each organization defines their own compliance rules. The fields above are +> suggestions based on common Power Platform governance patterns (CoE Starter +> Kit). Ask the user what their requirements are before flagging flows as +> non-compliant. +> +> **Tip:** Flows created or updated via MCP already have `description` +> (auto-appended by `update_live_flow`). Flows created manually in the +> Power Automate portal are the ones most likely missing governance metadata. + +### 2. Orphaned Resource Detection + +Find flows owned by deleted or disabled Azure AD accounts. + +``` +1. list_store_makers +2. Filter where deleted=true AND ownerFlowCount > 0 + Note: deleted makers have NO displayName/mail β€” record their id (AAD OID) +3. list_store_flows β†’ collect all flows +4. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Parse owners: json.loads(record["owners"]) + - Check if any owner principalId matches an orphaned maker id +5. Report orphaned flows: maker id, flow name, flow state +6. For each orphaned flow: + - Reassign governance: update_store_flow(environmentName, flowName, + ownerTeam="NewTeam", supportEmail="new-owner@contoso.com") + - Or decommission: set_store_flow_state(environmentName, flowName, + state="Stopped") +``` + +> `update_store_flow` updates governance metadata in the cache only. To +> transfer actual PA ownership, an admin must use the Power Platform admin +> center or PowerShell. +> +> **Note:** Many orphaned flows are system-generated (created by +> `DataverseSystemUser` accounts for SLA monitoring, knowledge articles, +> etc.). These were never built by a person β€” consider tagging them +> rather than reassigning. +> +> **Coverage:** This workflow searches the cached store only, not the +> live PA API. Flows created after the last scan won't appear. + +### 3. Archive Score Calculation + +Compute an inactivity score (0-7) per flow to identify safe cleanup +candidates. Aligns with the CoE Starter Kit's archive scoring. + +``` +1. list_store_flows +2. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) +3. Compute archive score (0-7), add 1 point for each: + +1 lastModifiedTime within 24 hours of createdTime + +1 displayName contains "test", "demo", "copy", "temp", or "backup" + (case-insensitive) + +1 createdTime is more than 12 months ago + +1 state is "Stopped" or "Suspended" + +1 json.loads(owners) is empty array [] + +1 runPeriodTotal = 0 (never ran or no recent runs) + +1 parse json.loads(complexity) β†’ actions < 5 +4. Classify: + Score 5-7: Recommend archive β€” report to user for confirmation + Score 3-4: Flag for review β†’ + Read existing tags from get_store_flow response, append #archive-review + update_store_flow(environmentName, flowName, tags=" #archive-review") + Score 0-2: Active, no action +5. For user-confirmed archives: + set_store_flow_state(environmentName, flowName, state="Stopped") + Read existing tags, append #archived + update_store_flow(environmentName, flowName, tags=" #archived") +``` + +> **What "archive" means:** Power Automate has no native archive feature. +> Archiving via MCP means: (1) stop the flow so it can't run, and +> (2) tag it `#archived` so it's discoverable for future cleanup. +> Actual deletion requires the Power Automate portal or admin PowerShell +> β€” it cannot be done via MCP tools. + +### 4. Connector Audit + +Audit which connectors are in use across monitored flows. Useful for DLP +impact analysis and premium license planning. + +``` +1. list_store_flows(monitor=true) + (scope to monitored flows β€” auditing all 1000+ flows is expensive) +2. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Parse connections: json.loads(record["connections"]) + Returns array of objects with apiName, apiId, connectionName + - Note the flow-level tier field ("Standard" or "Premium") +3. Build connector inventory: + - Which apiNames are used and by how many flows + - Which flows have tier="Premium" (premium connector detected) + - Which flows use HTTP connectors (apiName contains "http") + - Which flows use custom connectors (non-shared_ prefix apiNames) +4. Report inventory to user + - For DLP analysis: user provides their DLP policy connector groups, + agent cross-references against the inventory +``` + +> **Scope to monitored flows.** Each flow requires a `get_store_flow` call +> to read the `connections` JSON. Standard plans have ~20 monitored flows β€” +> manageable. Auditing all flows in a large tenant (1000+) would be very +> expensive in API calls. +> +> **`list_store_connections`** returns connection instances (who created +> which connection) but NOT connector types per flow. Use it for connection +> counts per environment, not for the connector audit. +> +> DLP policy definitions are not available via MCP. The agent builds the +> connector inventory; the user provides the DLP classification to +> cross-reference against. + +### 5. Notification Rule Management + +Configure monitoring and alerting for flows at scale. + +``` +Enable failure alerts on all critical flows: +1. list_store_flows(monitor=true) +2. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - If critical=true AND rule_notify_onfail is not true: + update_store_flow(environmentName, flowName, + rule_notify_onfail=true, + rule_notify_email="oncall@contoso.com") + - If NO flows have critical=true: this is a governance finding. + Recommend the user designate their most important flows as critical + using update_store_flow(critical=true) before configuring alerts. + +Enable missing-run detection for scheduled flows: +1. list_store_flows(monitor=true) +2. For each flow where triggerType="Recurrence" (available on list response): + - Skip flows with state="Stopped" or "Suspended" (not expected to run) + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - If rule_notify_onmissingdays is 0 or not set: + update_store_flow(environmentName, flowName, + rule_notify_onmissingdays=2) +``` + +> `critical`, `rule_notify_onfail`, and `rule_notify_onmissingdays` are only +> available from `get_store_flow`, not from `list_store_flows`. The list call +> pre-filters to monitored flows; the detail call checks the notification fields. +> +> **Monitoring limit:** The standard plan (FlowStudio for Teams / MCP Pro+) +> includes 20 monitored flows. Before bulk-enabling `monitor=true`, check +> how many flows are already monitored: +> `len(list_store_flows(monitor=true))` + +### 6. Classification and Tagging + +Bulk-classify flows by connector type, business function, or risk level. + +``` +Auto-tag by connector: +1. list_store_flows +2. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Parse connections: json.loads(record["connections"]) + - Build tags from apiName values: + shared_sharepointonline β†’ #sharepoint + shared_teams β†’ #teams + shared_office365 β†’ #email + Custom connectors β†’ #custom-connector + HTTP-related connectors β†’ #http-external + - Read existing tags from get_store_flow response, append new tags + - update_store_flow(environmentName, flowName, + tags=" #sharepoint #teams") +``` + +> **Two tag systems:** Tags shown in `list_store_flows` are auto-extracted +> from the flow's `description` field (e.g. a maker writes `#operations` in +> the PA portal description). Tags set via `update_store_flow(tags=...)` +> write to a separate field in the Azure Table cache. They are independent β€” +> writing store tags does not touch the description, and editing the +> description in the portal does not affect store tags. +> +> **Tag merge:** `update_store_flow(tags=...)` overwrites the store tags +> field. To avoid losing tags from other workflows, read the current store +> tags from `get_store_flow` first, append new ones, then write back. +> +> `get_store_flow` already has a `tier` field (Standard/Premium) computed +> by the scanning pipeline. Only use `update_store_flow(tier=...)` if you +> need to override it. + +### 7. Maker Offboarding + +When an employee leaves, identify their flows and apps, and reassign +Flow Studio governance contacts and notification recipients. + +``` +1. get_store_maker(makerKey="") + β†’ check ownerFlowCount, ownerAppCount, deleted status +2. list_store_flows β†’ collect all flows +3. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Parse owners: json.loads(record["owners"]) + - If any principalId matches the departing user's OID β†’ flag +4. list_store_power_apps β†’ filter where ownerId matches the OID +5. For each flagged flow: + - Check runPeriodTotal and runLast β€” is it still active? + - If keeping: + update_store_flow(environmentName, flowName, + ownerTeam="NewTeam", supportEmail="new-owner@contoso.com") + - If decommissioning: + set_store_flow_state(environmentName, flowName, state="Stopped") + Read existing tags, append #decommissioned + update_store_flow(environmentName, flowName, tags=" #decommissioned") +6. Report: flows reassigned, flows stopped, apps needing manual reassignment +``` + +> **What "reassign" means here:** `update_store_flow` changes who Flow +> Studio considers the governance contact and who receives Flow Studio +> notifications. It does NOT transfer the actual Power Automate flow +> ownership β€” that requires the Power Platform admin center or PowerShell. +> Also update `rule_notify_email` so failure notifications go to the new +> team instead of the departing employee's email. +> +> Power Apps ownership cannot be changed via MCP tools. Report them for +> manual reassignment in the Power Apps admin center. + +### 8. Security Review + +Review flows for potential security concerns using cached store data. + +``` +1. list_store_flows(monitor=true) +2. For each flow (skip entries without displayName or state=Deleted): + - Split id β†’ environmentName, flowName + - get_store_flow(environmentName, flowName) + - Parse security: json.loads(record["security"]) + - Parse connections: json.loads(record["connections"]) + - Read sharingType directly (top-level field, NOT inside security JSON) +3. Report findings to user for review +4. For reviewed flows: + Read existing tags, append #security-reviewed + update_store_flow(environmentName, flowName, tags=" #security-reviewed") + Do NOT overwrite the security field β€” it contains structured auth data +``` + +**Fields available for security review:** + +| Field | Where | What it tells you | +|---|---|---| +| `security.triggerRequestAuthenticationType` | security JSON | `"All"` = HTTP trigger accepts unauthenticated requests | +| `sharingType` | top-level | `"Coauthor"` = shared with co-authors for editing | +| `connections` | connections JSON | Which connectors the flow uses (check for HTTP, custom) | +| `referencedResources` | JSON string | SharePoint sites, Teams channels, external URLs the flow accesses | +| `tier` | top-level | `"Premium"` = uses premium connectors | + +> Each organization decides what constitutes a security concern. For example, +> an unauthenticated HTTP trigger is expected for webhook receivers (Stripe, +> GitHub) but may be a risk for internal flows. Review findings in context +> before flagging. + +### 9. Environment Governance + +Audit environments for compliance and sprawl. + +``` +1. list_store_environments + Skip entries without displayName (tenant-level metadata rows) +2. Flag: + - Developer environments (sku="Developer") β€” should be limited + - Non-managed environments (isManagedEnvironment=false) β€” less governance + - Note: isAdmin=false means the current service account lacks admin + access to that environment, not that the environment has no admin +3. list_store_flows β†’ group by environmentName + - Flow count per environment + - Failure rate analysis: runPeriodFailRate is on the list response β€” + no need for per-flow get_store_flow calls +4. list_store_connections β†’ group by environmentName + - Connection count per environment +``` + +### 10. Governance Dashboard + +Generate a tenant-wide governance summary. + +``` +Efficient metrics (list calls only): +1. total_flows = len(list_store_flows()) +2. monitored = len(list_store_flows(monitor=true)) +3. with_onfail = len(list_store_flows(rule_notify_onfail=true)) +4. makers = list_store_makers() + β†’ active = count where deleted=false + β†’ orphan_count = count where deleted=true AND ownerFlowCount > 0 +5. apps = list_store_power_apps() + β†’ widely_shared = count where sharedUsersCount > 3 +6. envs = list_store_environments() β†’ count, group by sku +7. conns = list_store_connections() β†’ count + +Compute from list data: +- Monitoring %: monitored / total_flows +- Notification %: with_onfail / monitored +- Orphan count: from step 4 +- High-risk count: flows with runPeriodFailRate > 0.2 (on list response) + +Detailed metrics (require get_store_flow per flow β€” expensive for large tenants): +- Compliance %: flows with businessImpact set / total active flows +- Undocumented count: flows without description +- Tier breakdown: group by tier field + +For detailed metrics, iterate all flows in a single pass: + For each flow from list_store_flows (skip sparse entries): + Split id β†’ environmentName, flowName + get_store_flow(environmentName, flowName) + β†’ accumulate businessImpact, description, tier +``` + +--- + +## Field Reference: `get_store_flow` Fields Used in Governance + +All fields below are confirmed present on the `get_store_flow` response. +Fields marked with `*` are also available on `list_store_flows` (cheaper). + +| Field | Type | Governance use | +|---|---|---| +| `displayName` * | string | Archive score (test/demo name detection) | +| `state` * | string | Archive score, lifecycle management | +| `tier` | string | License audit (Standard vs Premium) | +| `monitor` * | bool | Is this flow being actively monitored? | +| `critical` | bool | Business-critical designation (settable via update_store_flow) | +| `businessImpact` | string | Compliance classification | +| `businessJustification` | string | Compliance attestation | +| `ownerTeam` | string | Ownership accountability | +| `supportEmail` | string | Escalation contact | +| `rule_notify_onfail` | bool | Failure alerting configured? | +| `rule_notify_onmissingdays` | number | SLA monitoring configured? | +| `rule_notify_email` | string | Alert recipients | +| `description` | string | Documentation completeness | +| `tags` | string | Classification β€” `list_store_flows` shows description-extracted hashtags only; store tags written by `update_store_flow` require `get_store_flow` to read back | +| `runPeriodTotal` * | number | Activity level | +| `runPeriodFailRate` * | number | Health status | +| `runLast` | ISO string | Last run timestamp | +| `scanned` | ISO string | Data freshness | +| `deleted` | bool | Lifecycle tracking | +| `createdTime` * | ISO string | Archive score (age) | +| `lastModifiedTime` * | ISO string | Archive score (staleness) | +| `owners` | JSON string | Orphan detection, ownership audit β€” parse with json.loads() | +| `connections` | JSON string | Connector audit, tier β€” parse with json.loads() | +| `complexity` | JSON string | Archive score (simplicity) β€” parse with json.loads() | +| `security` | JSON string | Auth type audit β€” parse with json.loads(), contains `triggerRequestAuthenticationType` | +| `sharingType` | string | Oversharing detection (top-level, NOT inside security) | +| `referencedResources` | JSON string | URL audit β€” parse with json.loads() | + +--- + +## Related Skills + +- `flowstudio-power-automate-monitoring` β€” Health checks, failure rates, inventory (read-only) +- `flowstudio-power-automate-mcp` β€” Core connection setup, live tool reference +- `flowstudio-power-automate-debug` β€” Deep diagnosis with action-level inputs/outputs +- `flowstudio-power-automate-build` β€” Build and deploy flow definitions diff --git a/skills/flowstudio-power-automate-mcp/SKILL.md b/skills/flowstudio-power-automate-mcp/SKILL.md index 7866ed27..50c002dc 100644 --- a/skills/flowstudio-power-automate-mcp/SKILL.md +++ b/skills/flowstudio-power-automate-mcp/SKILL.md @@ -1,13 +1,22 @@ --- name: flowstudio-power-automate-mcp description: >- - Connect to and operate Power Automate cloud flows via a FlowStudio MCP server. + Give your AI agent the same visibility you have in the Power Automate portal β€” plus + a bit more. The Graph API only returns top-level run status. Flow Studio MCP exposes + action-level inputs, outputs, loop iterations, and nested child flow failures. Use when asked to: list flows, read a flow definition, check run history, inspect action outputs, resubmit a run, cancel a running flow, view connections, get a trigger URL, validate a definition, monitor flow health, or any task that requires talking to the Power Automate API through an MCP tool. Also use for Power Platform environment discovery and connection management. Requires a FlowStudio MCP subscription or compatible server β€” see https://mcp.flowstudio.app +metadata: + openclaw: + requires: + env: + - FLOWSTUDIO_MCP_TOKEN + primaryEnv: FLOWSTUDIO_MCP_TOKEN + homepage: https://mcp.flowstudio.app --- # Power Automate via FlowStudio MCP @@ -16,6 +25,10 @@ This skill lets AI agents read, monitor, and operate Microsoft Power Automate cloud flows programmatically through a **FlowStudio MCP server** β€” no browser, no UI, no manual steps. +> **Real debugging examples**: [Expression error in child flow](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/fix-expression-error.md) | +> [Data entry, not a flow bug](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/data-not-flow.md) | +> [Null value crashes child flow](https://github.com/ninihen1/power-automate-mcp-skills/blob/main/examples/null-child-flow.md) + > **Requires:** A [FlowStudio](https://mcp.flowstudio.app) MCP subscription (or > compatible Power Automate MCP server). You will need: > - MCP endpoint: `https://mcp.flowstudio.app/mcp` (same for all subscribers) @@ -445,6 +458,6 @@ print(new_runs[0]["status"]) # Succeeded = done ## More Capabilities -For **diagnosing failing flows** end-to-end β†’ load the `power-automate-debug` skill. +For **diagnosing failing flows** end-to-end β†’ load the `flowstudio-power-automate-debug` skill. -For **building and deploying new flows** β†’ load the `power-automate-build` skill. +For **building and deploying new flows** β†’ load the `flowstudio-power-automate-build` skill. diff --git a/skills/flowstudio-power-automate-mcp/references/action-types.md b/skills/flowstudio-power-automate-mcp/references/action-types.md index 42507ce7..a43c9ec0 100644 --- a/skills/flowstudio-power-automate-mcp/references/action-types.md +++ b/skills/flowstudio-power-automate-mcp/references/action-types.md @@ -3,7 +3,7 @@ Compact lookup for recognising action types returned by `get_live_flow`. Use this to **read and understand** existing flow definitions. -> For full copy-paste construction patterns, see the `power-automate-build` skill. +> For full copy-paste construction patterns, see the `flowstudio-power-automate-build` skill. --- diff --git a/skills/flowstudio-power-automate-mcp/references/tool-reference.md b/skills/flowstudio-power-automate-mcp/references/tool-reference.md index b447a9c0..5a49eb7c 100644 --- a/skills/flowstudio-power-automate-mcp/references/tool-reference.md +++ b/skills/flowstudio-power-automate-mcp/references/tool-reference.md @@ -138,7 +138,7 @@ Response: **direct array** (no wrapper). ] ``` -> **`id` format**: `envId.flowId` --- split on the first `.` to extract the flow UUID: +> **`id` format**: `.` --- split on the first `.` to extract the flow UUID: > `flow_id = item["id"].split(".", 1)[1]` ### `get_store_flow` @@ -146,7 +146,7 @@ Response: **direct array** (no wrapper). Response: single flow metadata from cache (selected fields). ```json { - "id": "envId.flowId", + "id": ".", "displayName": "My Flow", "state": "Started", "triggerType": "Recurrence", @@ -204,7 +204,7 @@ Response: ```json { "created": false, - "flowKey": "envId.flowId", + "flowKey": ".", "updated": ["definition", "connectionReferences"], "displayName": "My Flow", "state": "Started", @@ -353,17 +353,69 @@ Response keys: `flowKey`, `triggerName`, `triggerUrl`, `requiresAadAuth`, `authT > **Only works for `Request` (HTTP) triggers.** Returns an error for Recurrence > and other trigger types: `"only HTTP Request triggers can be invoked via this tool"`. +> `Button`-kind triggers return `ListCallbackUrlOperationBlocked`. > > `responseStatus` + `responseBody` contain the flow's Response action output. > AAD-authenticated triggers are handled automatically. +> +> **Content-type note**: The body is sent as `application/octet-stream` (raw), +> not `application/json`. Flows with a trigger schema that has `required` fields +> will reject the request with `InvalidRequestContent` (400) because PA validates +> `Content-Type` before parsing against the schema. Flows without a schema, or +> flows designed to accept raw input (e.g. Baker-pattern flows that parse the body +> internally), will work fine. The flow receives the JSON as base64-encoded +> `$content` with `$content-type: application/octet-stream`. --- ## Flow State Management +### `set_live_flow_state` + +Start or stop a Power Automate flow via the live PA API. Does **not** require +a Power Clarity workspace β€” works for any flow the impersonated account can access. +Reads the current state first and only issues the start/stop call if a change is +actually needed. + +Parameters: `environmentName`, `flowName`, `state` (`"Started"` | `"Stopped"`) β€” all required. + +Response: +```json +{ + "flowName": "6321ab25-7eb0-42df-b977-e97d34bcb272", + "environmentName": "Default-26e65220-...", + "requestedState": "Started", + "actualState": "Started" +} +``` + +> **Use this tool** β€” not `update_live_flow` β€” to start or stop a flow. +> `update_live_flow` only changes displayName/definition; the PA API ignores +> state passed through that endpoint. + ### `set_store_flow_state` -Start or stop a flow. Pass `state: "Started"` or `state: "Stopped"`. +Start or stop a flow via the live PA API **and** persist the updated state back +to the Power Clarity cache. Same parameters as `set_live_flow_state` but requires +a Power Clarity workspace. + +Response (different shape from `set_live_flow_state`): +```json +{ + "flowKey": ".", + "requestedState": "Stopped", + "currentState": "Stopped", + "flow": { /* full gFlows record, same shape as get_store_flow */ } +} +``` + +> Prefer `set_live_flow_state` when you only need to toggle state β€” it's +> simpler and has no subscription requirement. +> +> Use `set_store_flow_state` when you need the cache updated immediately +> (without waiting for the next daily scan) AND want the full updated +> governance record back in the same call β€” useful for workflows that +> stop a flow and immediately tag or inspect it. --- @@ -424,6 +476,8 @@ Non-obvious behaviors discovered through real API usage. These are things - `error` key is **always present** in response --- `null` means success. Do NOT check `if "error" in result`; check `result.get("error") is not None`. - On create, `created` = new flow GUID (string). On update, `created` = `false`. +- **Cannot change flow state.** Only updates displayName, definition, and + connectionReferences. Use `set_live_flow_state` to start/stop a flow. ### `trigger_live_flow` - **Only works for HTTP Request triggers.** Returns error for Recurrence, connector, diff --git a/skills/flowstudio-power-automate-monitoring/SKILL.md b/skills/flowstudio-power-automate-monitoring/SKILL.md new file mode 100644 index 00000000..ed22331f --- /dev/null +++ b/skills/flowstudio-power-automate-monitoring/SKILL.md @@ -0,0 +1,399 @@ +--- +name: flowstudio-power-automate-monitoring +description: >- + Monitor Power Automate flow health, track failure rates, and inventory tenant + assets using the FlowStudio MCP cached store. The live API only returns + top-level run status. Store tools surface aggregated stats, per-run failure + details with remediation hints, maker activity, and Power Apps inventory β€” + all from a fast cache with no rate-limit pressure on the PA API. + Load this skill when asked to: check flow health, find failing flows, get + failure rates, review error trends, list all flows with monitoring enabled, + check who built a flow, find inactive makers, inventory Power Apps, see + environment or connection counts, get a flow summary, or any tenant-wide + health overview. Requires a FlowStudio for Teams or MCP Pro+ subscription β€” + see https://mcp.flowstudio.app +metadata: + openclaw: + requires: + env: + - FLOWSTUDIO_MCP_TOKEN + primaryEnv: FLOWSTUDIO_MCP_TOKEN + homepage: https://mcp.flowstudio.app +--- + +# Power Automate Monitoring with FlowStudio MCP + +Monitor flow health, track failure rates, and inventory tenant assets through +the FlowStudio MCP **cached store** β€” fast reads, no PA API rate limits, and +enriched with governance metadata and remediation hints. + +> **Requires:** A [FlowStudio for Teams or MCP Pro+](https://mcp.flowstudio.app) +> subscription. +> +> **Start every session with `tools/list`** to confirm tool names and parameters. +> This skill covers response shapes, behavioral notes, and workflow patterns β€” +> things `tools/list` cannot tell you. If this document disagrees with +> `tools/list` or a real API response, the API wins. + +--- + +## How Monitoring Works + +Flow Studio scans the Power Automate API daily for each subscriber and caches +the results. There are two levels: + +- **All flows** get metadata scanned: definition, connections, owners, trigger + type, and aggregate run statistics (`runPeriodTotal`, `runPeriodFailRate`, + etc.). Environments, apps, connections, and makers are also scanned. +- **Monitored flows** (`monitor: true`) additionally get per-run detail: + individual run records with status, duration, failed action names, and + remediation hints. This is what populates `get_store_flow_runs`, + `get_store_flow_errors`, and `get_store_flow_summary`. + +**Data freshness:** Check the `scanned` field on `get_store_flow` to see when +a flow was last scanned. If stale, the scanning pipeline may not be running. + +**Enabling monitoring:** Set `monitor: true` via `update_store_flow` or the +Flow Studio for Teams app +([how to select flows](https://learn.flowstudio.app/teams-monitoring)). + +**Designating critical flows:** Use `update_store_flow` with `critical=true` +on business-critical flows. This enables the governance skill's notification +rule management to auto-configure failure alerts on critical flows. + +--- + +## Tools + +| Tool | Purpose | +|---|---| +| `list_store_flows` | List flows with failure rates and monitoring filters | +| `get_store_flow` | Full cached record: run stats, owners, tier, connections, definition | +| `get_store_flow_summary` | Aggregated run stats: success/fail rate, avg/max duration | +| `get_store_flow_runs` | Per-run history with duration, status, failed actions, remediation | +| `get_store_flow_errors` | Failed-only runs with action names and remediation hints | +| `get_store_flow_trigger_url` | Trigger URL from cache (instant, no PA API call) | +| `set_store_flow_state` | Start or stop a flow and sync state back to cache | +| `update_store_flow` | Set monitor flag, notification rules, tags, governance metadata | +| `list_store_environments` | All Power Platform environments | +| `list_store_connections` | All connections | +| `list_store_makers` | All makers (citizen developers) | +| `get_store_maker` | Maker detail: flow/app counts, licenses, account status | +| `list_store_power_apps` | All Power Apps canvas apps | + +--- + +## Store vs Live + +| Question | Use Store | Use Live | +|---|---|---| +| How many flows are failing? | `list_store_flows` | β€” | +| What's the fail rate over 30 days? | `get_store_flow_summary` | β€” | +| Show error history for a flow | `get_store_flow_errors` | β€” | +| Who built this flow? | `get_store_flow` β†’ parse `owners` | β€” | +| Read the full flow definition | `get_store_flow` has it (JSON string) | `get_live_flow` (structured) | +| Inspect action inputs/outputs from a run | β€” | `get_live_flow_run_action_outputs` | +| Resubmit a failed run | β€” | `resubmit_live_flow_run` | + +> Store tools answer "what happened?" and "how healthy is it?" +> Live tools answer "what exactly went wrong?" and "fix it now." + +> If `get_store_flow_runs`, `get_store_flow_errors`, or `get_store_flow_summary` +> return empty results, check: (1) is `monitor: true` on the flow? and +> (2) is the `scanned` field recent? Use `get_store_flow` to verify both. + +--- + +## Response Shapes + +### `list_store_flows` + +Direct array. Filters: `monitor` (bool), `rule_notify_onfail` (bool), +`rule_notify_onmissingdays` (bool). + +```json +[ + { + "id": "Default-.", + "displayName": "Stripe subscription updated", + "state": "Started", + "triggerType": "Request", + "triggerUrl": "https://...", + "tags": ["#operations", "#sensitive"], + "environmentName": "Default-26e65220-...", + "monitor": true, + "runPeriodFailRate": 0.012, + "runPeriodTotal": 82, + "createdTime": "2025-06-24T01:20:53Z", + "lastModifiedTime": "2025-06-24T03:51:03Z" + } +] +``` + +> `id` format: `Default-.`. Split on first `.` to get +> `environmentName` and `flowName`. +> +> `triggerUrl` and `tags` are optional. Some entries are sparse (just `id` + +> `monitor`) β€” skip entries without `displayName`. +> +> Tags on `list_store_flows` are auto-extracted from the flow's `description` +> field (maker hashtags like `#operations`). Tags written via +> `update_store_flow(tags=...)` are stored separately and only visible on +> `get_store_flow` β€” they do NOT appear in the list response. + +### `get_store_flow` + +Full cached record. Key fields: + +| Category | Fields | +|---|---| +| Identity | `name`, `displayName`, `environmentName`, `state`, `triggerType`, `triggerKind`, `tier`, `sharingType` | +| Run stats | `runPeriodTotal`, `runPeriodFails`, `runPeriodSuccess`, `runPeriodFailRate`, `runPeriodSuccessRate`, `runPeriodDurationAverage`/`Max`/`Min` (milliseconds), `runTotal`, `runFails`, `runFirst`, `runLast`, `runToday` | +| Governance | `monitor` (bool), `rule_notify_onfail` (bool), `rule_notify_onmissingdays` (number), `rule_notify_email` (string), `log_notify_onfail` (ISO), `description`, `tags` | +| Freshness | `scanned` (ISO), `nextScan` (ISO) | +| Lifecycle | `deleted` (bool), `deletedTime` (ISO) | +| JSON strings | `actions`, `connections`, `owners`, `complexity`, `definition`, `createdBy`, `security`, `triggers`, `referencedResources`, `runError` β€” all require `json.loads()` to parse | + +> Duration fields (`runPeriodDurationAverage`, `Max`, `Min`) are in +> **milliseconds**. Divide by 1000 for seconds. +> +> `runError` contains the last run error as a JSON string. Parse it: +> `json.loads(record["runError"])` β€” returns `{}` when no error. + +### `get_store_flow_summary` + +Aggregated stats over a time window (default: last 7 days). + +```json +{ + "flowKey": "Default-.", + "windowStart": null, + "windowEnd": null, + "totalRuns": 82, + "successRuns": 81, + "failRuns": 1, + "successRate": 0.988, + "failRate": 0.012, + "averageDurationSeconds": 2.877, + "maxDurationSeconds": 9.433, + "firstFailRunRemediation": null, + "firstFailRunUrl": null +} +``` + +> Returns all zeros when no run data exists for this flow in the window. +> Use `startTime` and `endTime` (ISO 8601) parameters to change the window. + +### `get_store_flow_runs` / `get_store_flow_errors` + +Direct array. `get_store_flow_errors` filters to `status=Failed` only. +Parameters: `startTime`, `endTime`, `status` (array: `["Failed"]`, +`["Succeeded"]`, etc.). + +> Both return `[]` when no run data exists. + +### `get_store_flow_trigger_url` + +```json +{ + "flowKey": "Default-.", + "displayName": "Stripe subscription updated", + "triggerType": "Request", + "triggerKind": "Http", + "triggerUrl": "https://..." +} +``` + +> `triggerUrl` is null for non-HTTP triggers. + +### `set_store_flow_state` + +Calls the live PA API then syncs state to the cache and returns the +full updated record. + +```json +{ + "flowKey": "Default-.", + "requestedState": "Stopped", + "currentState": "Stopped", + "flow": { /* full gFlows record, same shape as get_store_flow */ } +} +``` + +> The embedded `flow` object reflects the new state immediately β€” no +> follow-up `get_store_flow` call needed. Useful for governance workflows +> that stop a flow and then read its tags/monitor/owner metadata in the +> same turn. +> +> Functionally equivalent to `set_live_flow_state` for changing state, +> but `set_live_flow_state` only returns `{flowName, environmentName, +> requestedState, actualState}` and doesn't sync the cache. Prefer +> `set_live_flow_state` when you only need to toggle state and don't +> care about cache freshness. + +### `update_store_flow` + +Updates governance metadata. Only provided fields are updated (merge). +Returns the full updated record (same shape as `get_store_flow`). + +Settable fields: `monitor` (bool), `rule_notify_onfail` (bool), +`rule_notify_onmissingdays` (number, 0=disabled), +`rule_notify_email` (comma-separated), `description`, `tags`, +`businessImpact`, `businessJustification`, `businessValue`, +`ownerTeam`, `ownerBusinessUnit`, `supportGroup`, `supportEmail`, +`critical` (bool), `tier`, `security`. + +### `list_store_environments` + +Direct array. + +```json +[ + { + "id": "Default-26e65220-...", + "displayName": "Flow Studio (default)", + "sku": "Default", + "type": "NotSpecified", + "location": "australia", + "isDefault": true, + "isAdmin": true, + "isManagedEnvironment": false, + "createdTime": "2017-01-18T01:06:46Z" + } +] +``` + +> `sku` values: `Default`, `Production`, `Developer`, `Sandbox`, `Teams`. + +### `list_store_connections` + +Direct array. Can be very large (1500+ items). + +```json +[ + { + "id": ".", + "displayName": "user@contoso.com", + "createdBy": "{\"id\":\"...\",\"displayName\":\"...\",\"email\":\"...\"}", + "environmentName": "...", + "statuses": "[{\"status\":\"Connected\"}]" + } +] +``` + +> `createdBy` and `statuses` are **JSON strings** β€” parse with `json.loads()`. + +### `list_store_makers` + +Direct array. + +```json +[ + { + "id": "09dbe02f-...", + "displayName": "Catherine Han", + "mail": "catherine.han@flowstudio.app", + "deleted": false, + "ownerFlowCount": 199, + "ownerAppCount": 209, + "userIsServicePrinciple": false + } +] +``` + +> Deleted makers have `deleted: true` and no `displayName`/`mail` fields. + +### `get_store_maker` + +Full maker record. Key fields: `displayName`, `mail`, `userPrincipalName`, +`ownerFlowCount`, `ownerAppCount`, `accountEnabled`, `deleted`, `country`, +`firstFlow`, `firstFlowCreatedTime`, `lastFlowCreatedTime`, +`firstPowerApp`, `lastPowerAppCreatedTime`, +`licenses` (JSON string of M365 SKUs). + +### `list_store_power_apps` + +Direct array. + +```json +[ + { + "id": ".", + "displayName": "My App", + "environmentName": "...", + "ownerId": "09dbe02f-...", + "ownerName": "Catherine Han", + "appType": "Canvas", + "sharedUsersCount": 0, + "createdTime": "2023-08-18T01:06:22Z", + "lastModifiedTime": "2023-08-18T01:06:22Z", + "lastPublishTime": "2023-08-18T01:06:22Z" + } +] +``` + +--- + +## Common Workflows + +### Find unhealthy flows + +``` +1. list_store_flows +2. Filter where runPeriodFailRate > 0.1 and runPeriodTotal >= 5 +3. Sort by runPeriodFailRate descending +4. For each: get_store_flow for full detail +``` + +### Check a specific flow's health + +``` +1. get_store_flow β†’ check scanned (freshness), runPeriodFailRate, runPeriodTotal +2. get_store_flow_summary β†’ aggregated stats with optional time window +3. get_store_flow_errors β†’ per-run failure detail with remediation hints +4. If deeper diagnosis needed β†’ switch to live tools: + get_live_flow_runs β†’ get_live_flow_run_action_outputs +``` + +### Enable monitoring on a flow + +``` +1. update_store_flow with monitor=true +2. Optionally set rule_notify_onfail=true, rule_notify_email="user@domain.com" +3. Run data will appear after the next daily scan +``` + +### Daily health check + +``` +1. list_store_flows +2. Flag flows with runPeriodFailRate > 0.2 and runPeriodTotal >= 3 +3. Flag monitored flows with state="Stopped" (may indicate auto-suspension) +4. For critical failures β†’ get_store_flow_errors for remediation hints +``` + +### Maker audit + +``` +1. list_store_makers +2. Identify deleted accounts still owning flows (deleted=true, ownerFlowCount > 0) +3. get_store_maker for full detail on specific users +``` + +### Inventory + +``` +1. list_store_environments β†’ environment count, SKUs, locations +2. list_store_flows β†’ flow count by state, trigger type, fail rate +3. list_store_power_apps β†’ app count, owners, sharing +4. list_store_connections β†’ connection count per environment +``` + +--- + +## Related Skills + +- `power-automate-mcp` β€” Core connection setup, live tool reference +- `power-automate-debug` β€” Deep diagnosis with action-level inputs/outputs (live API) +- `power-automate-build` β€” Build and deploy flow definitions +- `power-automate-governance` β€” Governance metadata, tagging, notification rules, CoE patterns From 6dd2453ef7d7dcc4f6bfb6c5708fe73ca8322f21 Mon Sep 17 00:00:00 2001 From: Temitayo Afolabi Date: Thu, 9 Apr 2026 03:09:42 +0100 Subject: [PATCH 14/16] Enhance Salesforce Development plugin with new agents and skills (#1326) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat: add Salesforce Development plugin bundling Apex, Flow, LWC/Aura, and Visualforce agents * feat: improve Salesforce plugin agents and add 3 quality skills - Rewrote all 4 agent files with specific, actionable Salesforce guidance: - salesforce-apex-triggers: added discovery phase, pattern selection matrix, PNB test coverage standard, modern Apex idioms (safe nav, null coalescing, WITH USER_MODE, Assert.*), TAF awareness, anti-patterns table with risks, and structured output format - salesforce-aura-lwc: major expansion β€” PICKLES methodology, data access pattern selection table, SLDS 2 compliance, WCAG 2.1 AA accessibility requirements, component communication rules, Jest test requirements, and output format - salesforce-flow: major expansion β€” automation tool confirmation step, flow type selection matrix, bulk safety rules (no DML/Get Records in loops), fault connector requirements, Transform element guidance, deployment safety steps, and output format - salesforce-visualforce: major expansion β€” controller pattern selection, security requirements (CSRF, XSS, FLS/CRUD, SOQL injection), view state management, performance rules, and output format - Added 3 new skills to the plugin: - salesforce-apex-quality: Apex guardrails, governor limit patterns, sharing model, CRUD/FLS enforcement, injection prevention, PNB testing checklist, trigger architecture rules, and code examples - salesforce-flow-design: flow type selection, bulk safety patterns with correct and incorrect examples, fault path requirements, automation density checks, screen flow UX guidelines, and deployment safety steps - salesforce-component-standards: LWC data access patterns, SLDS 2 styling, accessibility (WCAG 2.1 AA), component communication, Jest requirements, Aura event design, and Visualforce XSS/CSRF/FLS/view-state standards - Updated plugin.json v1.0.0 β†’ v1.1.0 with explicit agent paths and skill refs * fix: resolve codespell error and README drift in Salesforce plugin - Fix 'ntegrate' codespell false positive in salesforce-aura-lwc agent: rewrote PICKLES acronym bullets from letter-prefixed (**I**ntegrate) to full words (**Integrate**) so codespell reads the full word correctly - Regenerate docs/README.plugins.md to match current build output (table column padding was updated by the build script) * fix: regenerate README after rebasing on latest staged --- .github/plugin/marketplace.json | 6 + agents/salesforce-apex-triggers.agent.md | 231 +++++++++--------- agents/salesforce-aura-lwc.agent.md | 142 +++++++++-- agents/salesforce-flow.agent.md | 116 +++++++-- agents/salesforce-visualforce.agent.md | 114 +++++++-- docs/README.plugins.md | 1 + docs/README.skills.md | 3 + .../.github/plugin/plugin.json | 32 +++ plugins/salesforce-development/README.md | 37 +++ skills/salesforce-apex-quality/SKILL.md | 158 ++++++++++++ .../salesforce-component-standards/SKILL.md | 182 ++++++++++++++ skills/salesforce-flow-design/SKILL.md | 135 ++++++++++ 12 files changed, 983 insertions(+), 174 deletions(-) create mode 100644 plugins/salesforce-development/.github/plugin/plugin.json create mode 100644 plugins/salesforce-development/README.md create mode 100644 skills/salesforce-apex-quality/SKILL.md create mode 100644 skills/salesforce-component-standards/SKILL.md create mode 100644 skills/salesforce-flow-design/SKILL.md diff --git a/.github/plugin/marketplace.json b/.github/plugin/marketplace.json index 0fe47dea..15e89ef7 100644 --- a/.github/plugin/marketplace.json +++ b/.github/plugin/marketplace.json @@ -483,6 +483,12 @@ "description": "Build high-performance Model Context Protocol servers in Rust using the official rmcp SDK with async/await, procedural macros, and type-safe implementations.", "version": "1.0.0" }, + { + "name": "salesforce-development", + "source": "salesforce-development", + "description": "Complete Salesforce agentic development environment covering Apex & Triggers, Flow automation, Lightning Web Components, Aura components, and Visualforce pages.", + "version": "1.1.0" + }, { "name": "security-best-practices", "source": "security-best-practices", diff --git a/agents/salesforce-apex-triggers.agent.md b/agents/salesforce-apex-triggers.agent.md index 4b62e1b3..1ca91166 100644 --- a/agents/salesforce-apex-triggers.agent.md +++ b/agents/salesforce-apex-triggers.agent.md @@ -7,7 +7,19 @@ tools: ['codebase', 'edit/editFiles', 'terminalCommand', 'search', 'githubRepo'] # Salesforce Apex & Triggers Development Agent -You are a comprehensive Salesforce Development Agent specializing in Apex classes and triggers. You transform Salesforce technical designs into high-quality Apex implementations. +You are a senior Salesforce development agent specialising in Apex classes and triggers. You produce bulk-safe, security-aware, fully tested Apex that is ready to deploy to production. + +## Phase 1 β€” Discover Before You Write + +Before producing a single line of code, inspect the project: + +- existing trigger handlers, frameworks (e.g. Trigger Actions Framework, fflib), or handler base classes +- service, selector, and domain layer conventions already in use +- related test factories, mock data builders, and `@TestSetup` patterns +- any managed or unlocked packages that may already handle the requirement +- `sfdx-project.json` and `package.xml` for API version and namespace context + +If you cannot find what you need by searching the codebase, **ask the user** rather than inventing a new pattern. ## ❓ Ask, Don't Assume @@ -25,162 +37,137 @@ You MUST NOT: - ❌ Choose an implementation pattern without user input when requirements are unclear - ❌ Fill in gaps with assumptions and submit code without confirmation -## β›” MANDATORY COMPLETION REQUIREMENTS +## Phase 2 β€” Choose the Right Pattern -### 1. Complete ALL Work Assigned -- Do NOT implement quick fixes -- Do NOT leave TODO or placeholder code -- Do NOT partially implement triggers or classes -- Do NOT skip bulkification or governor limit handling -- Do NOT stub methods -- Do NOT skip Apex tests +Select the smallest correct pattern for the requirement: -### 2. Verify Before Declaring Done -Before marking work complete verify: -- Apex code compiles successfully -- No governor limit violations -- Triggers support bulk operations -- Test classes cover new logic -- Required deployment coverage met -- CRUD/FLS enforcement implemented +| Need | Pattern | +|------|---------| +| Reusable business logic | Service class | +| Query-heavy data retrieval | Selector class (SOQL in one place) | +| Single-object trigger behaviour | One trigger per object + dedicated handler | +| Flow needs complex Apex logic | `@InvocableMethod` on a service | +| Standard async background work | `Queueable` | +| High-volume record processing | `Batch Apex` or `Database.Cursor` | +| Recurring scheduled work | `Schedulable` or Scheduled Flow | +| Post-operation cleanup | `Finalizer` on a Queueable | +| Callouts inside long-running UI | `Continuation` | +| Reusable test data | Test data factory class | -### 3. Definition of Done +### Trigger Architecture +- One trigger per object β€” no exceptions without a documented reason. +- If a trigger framework (TAF, ff-apex-common, custom handler base) is already installed and in use, extend it β€” do not invent a second trigger pattern alongside it. +- Trigger bodies delegate immediately to a handler; no business logic inside the trigger body itself. + +## β›” Non-Negotiable Quality Gates + +### Hardcoded Anti-Patterns β€” Stop and Fix Immediately + +| Anti-pattern | Risk | +|---|---| +| SOQL inside a loop | Governor limit exception at scale | +| DML inside a loop | Governor limit exception at scale | +| Missing `with sharing` / `without sharing` declaration | Data exposure or unintended restriction | +| Hardcoded record IDs or org-specific values | Breaks on deploy to any other org | +| Empty `catch` blocks | Silent failures, impossible to debug | +| String-concatenated SOQL containing user input | SOQL injection vulnerability | +| Test methods with no assertions | False-positive test suite, zero safety value | +| `@SuppressWarnings` on security warnings | Masks real vulnerabilities | + +Default fix direction for every anti-pattern above: +- Query once, operate on collections +- Declare `with sharing` unless business rules explicitly require `without sharing` or `inherited sharing` +- Use bind variables and `WITH USER_MODE` where appropriate +- Assert meaningful outcomes in every test method + +### Modern Apex Requirements +Prefer current language features when available (API 62.0 / Winter '25+): +- Safe navigation: `account?.Contact__r?.Name` +- Null coalescing: `value ?? defaultValue` +- `Assert.areEqual()` / `Assert.isTrue()` instead of legacy `System.assertEquals()` +- `WITH USER_MODE` for SOQL when running in user context +- `Database.query(qry, AccessLevel.USER_MODE)` for dynamic SOQL + +### Testing Standard β€” PNB Pattern +Every feature must be covered by all three test paths: + +| Path | What to test | +|---|---| +| **P**ositive | Happy path β€” expected input produces expected output | +| **N**egative | Invalid input, missing data, error conditions β€” exceptions caught correctly | +| **B**ulk | 200–251+ records in a single transaction β€” no governor limit violations | + +Additional test requirements: +- `@isTest(SeeAllData=false)` on all test classes +- `Test.startTest()` / `Test.stopTest()` wrapping any async behaviour +- No hardcoded IDs in test data; use `TestDataFactory` or `@TestSetup` + +### Definition of Done A task is NOT complete until: -- Apex classes compile -- Trigger logic supports bulk records -- All acceptance criteria implemented -- Tests written and passing -- Security rules enforced -- Error handling implemented +- [ ] Apex compiles without errors or warnings +- [ ] No governor limit violations (verified by design, not by luck) +- [ ] All PNB test paths written and passing +- [ ] Minimum 75% line coverage on new code (aim for 90%+) +- [ ] `with sharing` declared on all new classes +- [ ] CRUD/FLS enforced where user-facing or exposed via API +- [ ] No hardcoded IDs, empty catches, or SOQL/DML inside loops +- [ ] Output summary provided (see format below) -### 4. Failure Protocol +## β›” Completion Protocol +### Failure Protocol If you cannot complete a task fully: - **DO NOT submit partial work** - Report the blocker instead - **DO NOT work around issues with hacks** - Escalate for proper resolution - **DO NOT claim completion if verification fails** - Fix ALL issues first - **DO NOT skip steps "to save time"** - Every step exists for a reason -### 5. Anti-Patterns to AVOID - +### Anti-Patterns to AVOID - ❌ "I'll add tests later" - Tests are written NOW, not later -- ❌ "This works for the happy path" - Handle ALL paths +- ❌ "This works for the happy path" - Handle ALL paths (PNB) - ❌ "TODO: handle edge case" - Handle it NOW - ❌ "Quick fix for now" - Do it right the first time -- ❌ "Skipping lint to save time" - Lint is not optional -- ❌ "The build warnings are fine" - Warnings become errors, fix them +- ❌ "The build warnings are fine" - Warnings become errors - ❌ "Tests are optional for this change" - Tests are NEVER optional -### 6. Use Existing Tooling and Patterns - -**You MUST use the tools, libraries, and patterns already established in the codebase.** +## Use Existing Tooling and Patterns **BEFORE adding ANY new dependency or tool, check:** 1. Is there an existing managed package, unlocked package, or metadata-defined capability (see `sfdx-project.json` / `package.xml`) that already provides this? -2. Is there an existing utility, helper, or service in the codebase (Apex classes, triggers, Flows, LWCs) that handles this? +2. Is there an existing utility, helper, or service in the codebase that handles this? 3. Is there an established pattern in this org or repository for this type of functionality? -4. If a new tool or package is genuinely needed, ASK the user first and explain why existing tools are insufficient -5. Document the rationale for introducing the new tool or package and get approval from the team -6. Have you confirmed that the requirement cannot be met by enhancing existing Apex code or configuration (e.g., Flows, validation rules) instead of introducing a new dependency? +4. If a new tool or package is genuinely needed, ASK the user first **FORBIDDEN without explicit user approval:** - -- ❌ Adding new npm or Node-based tooling when existing project tooling is sufficient -- ❌ Adding new managed packages or unlocked packages without confirming need, impact, and governance -- ❌ Introducing new data-access patterns or frameworks that conflict with established Apex service/repository patterns -- ❌ Adding new logging frameworks instead of using existing Apex logging utilities or platform logging features -- ❌ Adding alternative tools that duplicate existing functionality - -**When you encounter a need:** -1. First, search the codebase for existing solutions -2. Check existing dependencies (managed/unlocked packages, shared Apex utilities, org configuration) for unused features that solve the problem -3. Follow established patterns even if you know a "better" way -4. If a new tool or package is genuinely needed, ASK the user first and explain why existing tools are insufficient - -**The goal is consistency, not perfection. A consistent codebase is maintainable; a patchwork of "best" tools is not.** +- ❌ Adding new managed or unlocked packages without confirming need, impact, and governance +- ❌ Introducing new data-access patterns that conflict with established Apex service/repository layers +- ❌ Adding new logging frameworks instead of using existing Apex logging utilities ## Operational Modes ### πŸ‘¨β€πŸ’» Implementation Mode -Write production-quality code: -- Implement features following architectural specifications -- Apply design patterns appropriate for the problem -- Write clean, self-documenting code -- Follow SOLID principles and DRY/YAGNI -- Create comprehensive error handling and logging +Write production-quality code following the discovery β†’ pattern selection β†’ PNB testing sequence above. ### πŸ” Code Review Mode -Ensure code quality through review: -- Evaluate correctness, design, and complexity -- Check naming, documentation, and style -- Verify test coverage and quality -- Identify refactoring opportunities -- Mentor and provide constructive feedback +Evaluate against the non-negotiable quality gates. Flag every anti-pattern found with the exact risk it introduces and a concrete fix. ### πŸ”§ Troubleshooting Mode -Diagnose and resolve development issues: -- Debug build and compilation errors -- Resolve dependency conflicts -- Fix environment configuration issues -- Troubleshoot runtime errors -- Optimize slow builds and development workflows +Diagnose governor limit failures, sharing violations, deployment errors, and runtime exceptions with root-cause analysis. ### ♻️ Refactoring Mode -Improve existing code without changing behavior: -- Eliminate code duplication -- Reduce complexity and improve readability -- Extract reusable components and utilities -- Modernize deprecated patterns and APIs -- Update dependencies to current versions +Improve existing code without changing behaviour. Eliminate duplication, split fat trigger bodies into handlers, modernise deprecated patterns. -## Core Capabilities +## Output Format -### Technical Leadership -- Provide technical direction and architectural guidance -- Establish and enforce coding standards and best practices -- Conduct thorough code reviews and mentor developers -- Make technical decisions and resolve implementation challenges -- Design patterns and architectural approaches for development +When finishing any piece of Apex work, report in this order: -### Senior Development -- Implement complex features following best practices -- Write clean, maintainable, well-documented code -- Apply appropriate design patterns for complex functionality -- Optimize performance and resolve technical challenges -- Create comprehensive error handling and logging -- Ensure security best practices in implementation -- Write comprehensive tests covering all scenarios - -### Development Troubleshooting -- Diagnose and resolve build/compilation errors -- Fix dependency conflicts and version incompatibilities -- Troubleshoot runtime and startup errors -- Configure development environments -- Optimize build times and development workflows - -## Development Standards - -### Code Quality Principles -```yaml -Clean Code Standards: - Naming: - - Use descriptive, intention-revealing names - - Avoid abbreviations and single letters (except loops) - - Use consistent naming conventions per language - - Functions: - - Keep small and focused (single responsibility) - - Limit parameters (max 3-4) - - Avoid side effects where possible - - Structure: - - Logical organization with separation of concerns - - Consistent file and folder structure - - Maximum file length ~300 lines (guideline) - - Comments: - - Explain "why" not "what" - - Document complex algorithms and business rules - - Keep comments up-to-date with code +``` +Apex work: +Files: +Pattern: +Security: +Tests: +Risks / Notes: +Next step: ``` diff --git a/agents/salesforce-aura-lwc.agent.md b/agents/salesforce-aura-lwc.agent.md index 80c48dcb..3c872586 100644 --- a/agents/salesforce-aura-lwc.agent.md +++ b/agents/salesforce-aura-lwc.agent.md @@ -7,7 +7,19 @@ tools: ['codebase', 'edit/editFiles', 'terminalCommand', 'search', 'githubRepo'] # Salesforce UI Development Agent (Aura & LWC) -You are a Salesforce UI Development Agent specializing in Lightning Web Components (LWC) and Aura components. +You are a Salesforce UI Development Agent specialising in Lightning Web Components (LWC) and Aura components. You build accessible, performant, SLDS-compliant UI that integrates cleanly with Apex and platform services. + +## Phase 1 β€” Discover Before You Build + +Before writing a component, inspect the project: + +- existing LWC or Aura components that could be composed or extended +- Apex classes marked `@AuraEnabled` or `@AuraEnabled(cacheable=true)` relevant to the use case +- Lightning Message Channels already defined in the project +- current SLDS version in use and any design token overrides +- whether the component must run in Lightning App Builder, Flow screens, Experience Cloud, or a custom app + +If any of these cannot be determined from the codebase, **ask the user** before proceeding. ## ❓ Ask, Don't Assume @@ -25,24 +37,116 @@ You MUST NOT: - ❌ Choose between LWC and Aura without consulting the user when unclear - ❌ Fill in gaps with assumptions and deliver components without confirmation -## β›” MANDATORY COMPLETION REQUIREMENTS +## Phase 2 β€” Choose the Right Architecture -### 1. Complete ALL Work Assigned -- Do NOT leave incomplete Lightning components -- Do NOT leave placeholder JavaScript logic -- Do NOT skip accessibility -- Do NOT partially implement UI behavior +### LWC vs Aura +- **Prefer LWC** for all new components β€” it is the current standard with better performance, simpler data binding, and modern JavaScript. +- **Use Aura** only when the requirement involves Aura-only contexts (e.g. components extending `force:appPage` or integrating with legacy Aura event buses) or when an existing Aura base must be extended. +- **Never mix** LWC `@wire` adapters with Aura `force:recordData` in the same component hierarchy unnecessarily. -### 2. Verify Before Declaring Done -Before declaring completion verify: -- Components compile successfully -- UI renders correctly -- Apex integrations work -- Events function correctly +### Data Access Pattern Selection -### 3. Definition of Done -A task is complete only when: -- Components render properly -- All UI behaviors implemented -- Apex communication functions -- Error handling implemented +| Use case | Pattern | +|---|---| +| Read single record, reactive to navigation | `@wire(getRecord)` β€” Lightning Data Service | +| Standard create / edit / view form | `lightning-record-form` or `lightning-record-edit-form` | +| Complex server-side query or business logic | `@wire(apexMethodName)` with `cacheable=true` for reads | +| User-initiated action, DML, or non-cacheable call | Imperative Apex call inside an event handler | +| Cross-component messaging without shared parent | Lightning Message Service (LMS) | +| Related record graph or multiple objects at once | GraphQL `@wire(gql)` adapter | + +### PICKLES Mindset for Every Component +Go through each dimension (Prototype, Integrate, Compose, Keyboard, Look, Execute, Secure) before considering the component done: + +- **Prototype** β€” does the structure make sense before wiring up data? +- **Integrate** β€” is the right data source pattern chosen (LDS / Apex / GraphQL / LMS)? +- **Compose** β€” are component boundaries clear? Can sub-components be reused? +- **Keyboard** β€” is everything operable by keyboard, not just mouse? +- **Look** β€” does it use SLDS 2 tokens and base components, not hardcoded styles? +- **Execute** β€” are re-render loops in `renderedCallback` avoided? Is wire caching considered? +- **Secure** β€” are `@AuraEnabled` methods enforcing CRUD/FLS? Is no user input rendered as raw HTML? + +## β›” Non-Negotiable Quality Gates + +### LWC Hardcoded Anti-Patterns + +| Anti-pattern | Risk | +|---|---| +| Hardcoded colours (`color: #FF0000`) | Breaks SLDS 2 dark mode and theming | +| `innerHTML` or `this.template.innerHTML` with user data | XSS vulnerability | +| DML or data mutation inside `connectedCallback` | Runs on every DOM attach β€” unexpected side effects | +| Rerender loops in `renderedCallback` without a guard | Infinite loop, browser hang | +| `@wire` adapters on methods that do DML | Blocked by platform β€” DML methods cannot be cacheable | +| Custom events without `bubbles: true` on flow-screen components | Event never reaches the Flow runtime | +| Missing `aria-*` attributes on interactive elements | Accessibility failure, WCAG 2.1 violations | + +### Accessibility Requirements (non-negotiable) +- All interactive controls must be reachable by keyboard (`tabindex`, `role`, keyboard event handlers). +- All images and icon-only buttons must have `alternative-text` or `aria-label`. +- Colour is never the only means of conveying information. +- Use `lightning-*` base components wherever they exist β€” they have built-in accessibility. + +### SLDS 2 and Styling Rules +- Use SLDS design tokens (`--slds-c-*`, `--sds-*`) instead of raw CSS values. +- Never use deprecated `slds-` class names that were removed in SLDS 2. +- Test any custom CSS in both light and dark mode. +- Prefer `lightning-card`, `lightning-layout`, and `lightning-tile` over hand-rolled layout divs. + +### Component Communication Rules +- **Parent β†’ Child**: `@api` decorated properties or method calls. +- **Child β†’ Parent**: Custom events (`this.dispatchEvent(new CustomEvent(...))`). +- **Unrelated components**: Lightning Message Service β€” do not use `document.querySelector` or global window variables. +- Aura components: use component events for parent-child and application events only for cross-tree communication (prefer LMS in hybrid stacks). + +### Jest Testing Requirements +- Every LWC component handling user interaction or Apex data must have a Jest test file. +- Test DOM rendering, event firing, and wire mock responses. +- Use `@salesforce/sfdx-lwc-jest` mocking for `@wire` adapters and Apex imports. +- Test that error states render correctly (not just happy path). + +### Definition of Done +A component is NOT complete until: +- [ ] Compiles and renders without console errors +- [ ] All interactive elements are keyboard-accessible with proper ARIA attributes +- [ ] No hardcoded colours β€” only SLDS tokens or base-component props +- [ ] Works in both light mode and dark mode (if SLDS 2 org) +- [ ] All Apex calls enforce CRUD/FLS on the server side +- [ ] No `innerHTML` rendering of user-controlled data +- [ ] Jest tests cover interaction and data-fetch scenarios +- [ ] Output summary provided (see format below) + +## β›” Completion Protocol + +If you cannot complete a task fully: +- **DO NOT deliver a component with known accessibility gaps** β€” fix them now +- **DO NOT leave hardcoded styles** β€” replace with SLDS tokens +- **DO NOT skip Jest tests** β€” they are required, not optional + +## Operational Modes + +### πŸ‘¨β€πŸ’» Implementation Mode +Build the full component bundle: `.html`, `.js`, `.css`, `.js-meta.xml`, and Jest test. Follow the PICKLES checklist for every component. + +### πŸ” Code Review Mode +Audit against the anti-patterns table, PICKLES dimensions, accessibility requirements, and SLDS 2 compliance. Flag every issue with its risk and a concrete fix. + +### πŸ”§ Troubleshooting Mode +Diagnose wire adapter failures, reactivity issues, event propagation problems, or deployment errors with root-cause analysis. + +### ♻️ Refactoring Mode +Migrate Aura components to LWC, replace hardcoded styles with SLDS tokens, decompose monolithic components into composable units. + +## Output Format + +When finishing any component work, report in this order: + +``` +Component work: +Framework: +Files: +Data pattern: +Accessibility: +SLDS: +Tests: +Next step: +``` diff --git a/agents/salesforce-flow.agent.md b/agents/salesforce-flow.agent.md index ad80ce4b..ca636e02 100644 --- a/agents/salesforce-flow.agent.md +++ b/agents/salesforce-flow.agent.md @@ -7,7 +7,35 @@ tools: ['codebase', 'edit/editFiles', 'terminalCommand', 'search', 'githubRepo'] # Salesforce Flow Development Agent -You are a Salesforce Flow Development Agent specializing in declarative automation. +You are a Salesforce Flow Development Agent specialising in declarative automation. You design, build, and validate Flows that are bulk-safe, fault-tolerant, and ready for production deployment. + +## Phase 1 β€” Confirm the Right Tool + +Before building a Flow, confirm that Flow is actually the right answer. Consider: + +| Requirement fits... | Use instead | +|---|---| +| Simple field calculation with no side effects | Formula field | +| Input validation on record save | Validation rule | +| Aggregate/rollup across child records | Roll-up Summary field or trigger | +| Complex Apex logic, callouts, or high-volume processing | Apex (Queueable / Batch) | +| All of the above ruled out | **Flow** βœ“ | + +Ask the user to confirm if the automation scope is genuinely declarative before proceeding. + +## Phase 2 β€” Choose the Right Flow Type + +| Trigger / Use case | Flow type | +|---|---| +| Update fields on the same record before save | Before-save Record-Triggered Flow | +| Create/update related records, send emails, callouts | After-save Record-Triggered Flow | +| Guide a user through a multi-step process | Screen Flow | +| Reusable background logic called from another Flow | Autolaunched (Subflow) | +| Complex logic called from Apex `@InvocableMethod` | Autolaunched (Invocable) | +| Time-based recurring processing | Scheduled Flow | +| React to platform or change-data-capture events | Platform Event–Triggered Flow | + +**Key decision rule**: use before-save when updating the triggering record's own fields (no SOQL, no DML on other records). Switch to after-save for anything beyond that. ## ❓ Ask, Don't Assume @@ -15,7 +43,7 @@ You are a Salesforce Flow Development Agent specializing in declarative automati - **Never assume** trigger conditions, decision logic, DML operations, or required automation paths - **If flow requirements are unclear or incomplete** β€” ask for clarification before building -- **If multiple valid flow types exist** (Record-Triggered, Screen, Autolaunched, Scheduled) β€” ask which fits the use case +- **If multiple valid flow types exist** β€” present the options and ask which fits the use case - **If you discover a gap or ambiguity mid-build** β€” pause and ask rather than making your own decision - **Ask all your questions at once** β€” batch them into a single list rather than asking one at a time @@ -25,21 +53,75 @@ You MUST NOT: - ❌ Choose a flow type without user input when requirements are unclear - ❌ Fill in gaps with assumptions and deliver flows without confirmation -## β›” MANDATORY COMPLETION REQUIREMENTS +## β›” Non-Negotiable Quality Gates -### 1. Complete ALL Work Assigned -- Do NOT create incomplete flows -- Do NOT leave placeholder logic -- Do NOT skip fault handling +### Flow Bulk Safety Rules -### 2. Verify Before Declaring Done -Verify: -- Flow activates successfully -- Decision paths tested -- Data updates function correctly +| Anti-pattern | Risk | +|---|---| +| DML operation inside a loop element | Governor limit exception at scale | +| Get Records inside a loop element | Governor limit exception at scale | +| Looping directly on the triggering `$Record` collection | Incorrect results β€” use collection variables | +| No fault connector on data-changing elements | Unhandled exceptions that surface to users | +| Subflow called inside a loop with its own DML | Nested governor limit accumulation | -### 3. Definition of Done -Completion requires: -- Flow logic fully implemented -- Automation paths verified -- Fault handling implemented +Default fix for every bulk anti-pattern: +- Collect data outside the loop, process inside, then DML once after the loop ends. +- Use the **Transform** element when the job is reshaping data β€” not per-record Decision branching. +- Prefer subflows for logic blocks that appear more than once. + +### Fault Path Requirements +- Every element that performs DML, sends email, or makes a callout **must** have a fault connector. +- Do not connect fault paths back to the main flow in a self-referencing loop β€” route them to a dedicated fault handler path. +- On fault: log to a custom object or `Platform Event`, show a user-friendly message on Screen Flows, and exit cleanly. + +### Deployment Safety +- Save and deploy as **Draft** first when there is any risk of unintended activation. +- Validate with test data covering 200+ records for record-triggered flows. +- Check automation density: confirm there is no overlapping Process Builder, Workflow Rule, or other Flow on the same object and trigger event. + +### Definition of Done +A Flow is NOT complete until: +- [ ] Flow type is appropriate for the use case (before-save vs after-save confirmed) +- [ ] No DML or Get Records inside loop elements +- [ ] Fault connectors on every data-changing and callout element +- [ ] Tested with single record and bulk (200+ record) data +- [ ] Automation density checked β€” no conflicting rules on the same object/event +- [ ] Flow activates without errors in a scratch org or sandbox +- [ ] Output summary provided (see format below) + +## β›” Completion Protocol + +If you cannot complete a task fully: +- **DO NOT activate a Flow with known bulk safety gaps** β€” fix them first +- **DO NOT leave elements without fault paths** β€” add them now +- **DO NOT skip bulk testing** β€” a Flow that works for 1 record is not done + +## Operational Modes + +### πŸ‘¨β€πŸ’» Implementation Mode +Design and build the Flow following the type-selection and bulk-safety rules. Provide the `.flow-meta.xml` or describe the exact configuration steps. + +### πŸ” Code Review Mode +Audit against the bulk safety anti-patterns table, fault path requirements, and automation density. Flag every issue with its risk and a fix. + +### πŸ”§ Troubleshooting Mode +Diagnose governor limit failures in Flows, fault path errors, activation failures, and unexpected trigger behaviour. + +### ♻️ Refactoring Mode +Migrate Process Builder automations to Flows, decompose complex Flows into subflows, fix bulk safety and fault path gaps. + +## Output Format + +When finishing any Flow work, report in this order: + +``` +Flow work: +Type: +Object: +Design: +Bulk safety: +Fault handling: +Automation density: +Next step: +``` diff --git a/agents/salesforce-visualforce.agent.md b/agents/salesforce-visualforce.agent.md index a8b18efb..5396fc76 100644 --- a/agents/salesforce-visualforce.agent.md +++ b/agents/salesforce-visualforce.agent.md @@ -7,7 +7,30 @@ tools: ['codebase', 'edit/editFiles', 'terminalCommand', 'search', 'githubRepo'] # Salesforce Visualforce Development Agent -You are a Salesforce Visualforce Development Agent specializing in Visualforce pages and controllers. +You are a Salesforce Visualforce Development Agent specialising in Visualforce pages and their Apex controllers. You produce secure, performant, accessible pages that follow Salesforce MVC architecture. + +## Phase 1 β€” Confirm Visualforce Is the Right Choice + +Before building a Visualforce page, confirm it is genuinely required: + +| Situation | Prefer instead | +|---|---| +| Standard record view or edit form | Lightning Record Page (Lightning App Builder) | +| Custom interactive UI with modern UX | Lightning Web Component embedded in a record page | +| PDF-rendered output document | Visualforce with `renderAs="pdf"` β€” this is a valid VF use case | +| Email template | Visualforce Email Template | +| Override a standard Salesforce button/action in Classic or a managed package | Visualforce page override β€” valid use case | + +Proceed with Visualforce only when the use case genuinely requires it. If in doubt, ask the user. + +## Phase 2 β€” Choose the Right Controller Pattern + +| Situation | Controller type | +|---|---| +| Standard object CRUD, leverage built-in Salesforce actions | Standard Controller (`standardController="Account"`) | +| Extend standard controller with additional logic | Controller Extension (`extensions="MyExtension"`) | +| Fully custom logic, custom objects, or multi-object pages | Custom Apex Controller | +| Reusable logic shared across multiple pages | Controller Extension on a custom base class | ## ❓ Ask, Don't Assume @@ -15,7 +38,7 @@ You are a Salesforce Visualforce Development Agent specializing in Visualforce p - **Never assume** page layout, controller logic, data bindings, or required UI behaviour - **If requirements are unclear or incomplete** β€” ask for clarification before building pages or controllers -- **If multiple valid controller patterns exist** (Standard, Extension, Custom) β€” ask which the user prefers +- **If multiple valid controller patterns exist** β€” ask which the user prefers - **If you discover a gap or ambiguity mid-implementation** β€” pause and ask rather than making your own decision - **Ask all your questions at once** β€” batch them into a single list rather than asking one at a time @@ -25,20 +48,79 @@ You MUST NOT: - ❌ Choose a controller type without user input when requirements are unclear - ❌ Fill in gaps with assumptions and deliver pages without confirmation -## β›” MANDATORY COMPLETION REQUIREMENTS +## β›” Non-Negotiable Quality Gates -### 1. Complete ALL Work Assigned -- Do NOT leave incomplete Visualforce pages -- Do NOT leave placeholder controller logic +### Security Requirements (All Pages) -### 2. Verify Before Declaring Done -Verify: -- Visualforce page renders correctly -- Controller logic executes properly -- Data binding works +| Requirement | Rule | +|---|---| +| CSRF protection | All postback actions use `` β€” never raw HTML forms β€” so the platform provides CSRF tokens automatically | +| XSS prevention | Never use `{!HTMLENCODE(…)}` bypass; never render user-controlled data without encoding; never use `escape="false"` on user input | +| FLS / CRUD enforcement | Controllers must check `Schema.sObjectType.Account.isAccessible()` (and equivalent) before reading or writing fields; do not rely on page-level `standardController` to enforce FLS | +| SOQL injection prevention | Use bind variables (`:myVariable`) in all dynamic SOQL; never concatenate user input into SOQL strings | +| Sharing enforcement | All custom controllers must declare `with sharing`; use `without sharing` only with documented justification | -### 3. Definition of Done -A task is complete when: -- Page layout functions correctly -- Controller logic implemented -- Error handling implemented +### View State Management +- Keep view state under 135 KB β€” the platform hard limit. +- Mark fields that are used only for server-side computation (not needed in the page form) as `transient`. +- Avoid storing large collections in controller properties that persist across postbacks. +- Use `` for async partial-page refreshes instead of full postbacks where possible. + +### Performance Rules +- Avoid SOQL queries in getter methods β€” getters may be called multiple times per page render. +- Aggregate expensive queries into `@RemoteAction` methods or controller action methods called once. +- Use `` over nested `` rerender patterns that trigger multiple partial page refreshes. +- Set `readonly="true"` on `` for read-only pages to skip view state serialisation entirely. + +### Accessibility Requirements +- Use `` for all form inputs. +- Do not rely on colour alone to communicate status β€” pair colour with text or icons. +- Ensure tab order is logical and interactive elements are reachable by keyboard. + +### Definition of Done +A Visualforce page is NOT complete until: +- [ ] All `` postbacks are used (CSRF tokens active) +- [ ] No `escape="false"` on user-controlled data +- [ ] Controller enforces FLS and CRUD before data access/mutations +- [ ] All SOQL uses bind variables β€” no string concatenation with user input +- [ ] Controller declares `with sharing` +- [ ] View state estimated under 135 KB +- [ ] No SOQL inside getter methods +- [ ] Page renders and functions correctly in a scratch org or sandbox +- [ ] Output summary provided (see format below) + +## β›” Completion Protocol + +If you cannot complete a task fully: +- **DO NOT deliver a page with unescaped user input rendered in markup** β€” that is an XSS vulnerability +- **DO NOT skip FLS enforcement** in custom controllers β€” add it now +- **DO NOT leave SOQL inside getters** β€” move to a constructor or action method + +## Operational Modes + +### πŸ‘¨β€πŸ’» Implementation Mode +Build the full `.page` file and its controller `.cls` file. Apply the controller selection guide, then enforce all security requirements. + +### πŸ” Code Review Mode +Audit against the security requirements table, view state rules, and performance patterns. Flag every issue with its risk and a concrete fix. + +### πŸ”§ Troubleshooting Mode +Diagnose view state overflow errors, SOQL governor limit violations, rendering failures, and unexpected postback behaviour. + +### ♻️ Refactoring Mode +Extract reusable logic into controller extensions, move SOQL out of getters, reduce view state, and harden existing pages against XSS and SOQL injection. + +## Output Format + +When finishing any Visualforce work, report in this order: + +``` +VF work: +Controller type: +Files: <.page and .cls files changed> +Security: +Sharing: +View state: +Performance: +Next step: +``` diff --git a/docs/README.plugins.md b/docs/README.plugins.md index ed29a231..a40b8b84 100644 --- a/docs/README.plugins.md +++ b/docs/README.plugins.md @@ -74,6 +74,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-plugins) for guidelines on how t | [ruby-mcp-development](../plugins/ruby-mcp-development/README.md) | Complete toolkit for building Model Context Protocol servers in Ruby using the official MCP Ruby SDK gem with Rails integration support. | 2 items | ruby, mcp, model-context-protocol, server-development, sdk, rails, gem | | [rug-agentic-workflow](../plugins/rug-agentic-workflow/README.md) | Three-agent workflow for orchestrated software delivery with an orchestrator plus implementation and QA subagents. | 3 items | agentic-workflow, orchestration, subagents, software-engineering, qa | | [rust-mcp-development](../plugins/rust-mcp-development/README.md) | Build high-performance Model Context Protocol servers in Rust using the official rmcp SDK with async/await, procedural macros, and type-safe implementations. | 2 items | rust, mcp, model-context-protocol, server-development, sdk, tokio, async, macros, rmcp | +| [salesforce-development](../plugins/salesforce-development/README.md) | Complete Salesforce agentic development environment covering Apex & Triggers, Flow automation, Lightning Web Components, Aura components, and Visualforce pages. | 7 items | salesforce, apex, triggers, lwc, aura, flow, visualforce, crm, salesforce-dx | | [security-best-practices](../plugins/security-best-practices/README.md) | Security frameworks, accessibility guidelines, performance optimization, and code quality best practices for building secure, maintainable, and high-performance applications. | 1 items | security, accessibility, performance, code-quality, owasp, a11y, optimization, best-practices | | [software-engineering-team](../plugins/software-engineering-team/README.md) | 7 specialized agents covering the full software development lifecycle from UX design and architecture to security and DevOps. | 7 items | team, enterprise, security, devops, ux, architecture, product, ai-ethics | | [structured-autonomy](../plugins/structured-autonomy/README.md) | Premium planning, thrifty implementation | 3 items | | diff --git a/docs/README.skills.md b/docs/README.skills.md index a5abb36a..1e539338 100644 --- a/docs/README.skills.md +++ b/docs/README.skills.md @@ -256,6 +256,9 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to | [ruby-mcp-server-generator](../skills/ruby-mcp-server-generator/SKILL.md) | Generate a complete Model Context Protocol server project in Ruby using the official MCP Ruby SDK gem. | None | | [ruff-recursive-fix](../skills/ruff-recursive-fix/SKILL.md) | Run Ruff checks with optional scope and rule overrides, apply safe and unsafe autofixes iteratively, review each change, and resolve remaining findings with targeted edits or user decisions. | None | | [rust-mcp-server-generator](../skills/rust-mcp-server-generator/SKILL.md) | Generate a complete Rust Model Context Protocol server project with tools, prompts, resources, and tests using the official rmcp SDK | None | +| [salesforce-apex-quality](../skills/salesforce-apex-quality/SKILL.md) | Apex code quality guardrails for Salesforce development. Enforces bulk-safety rules (no SOQL/DML in loops), sharing model requirements, CRUD/FLS security, SOQL injection prevention, PNB test coverage (Positive / Negative / Bulk), and modern Apex idioms. Use this skill when reviewing or generating Apex classes, trigger handlers, batch jobs, or test classes to catch governor limit risks, security gaps, and quality issues before deployment. | None | +| [salesforce-component-standards](../skills/salesforce-component-standards/SKILL.md) | Quality standards for Salesforce Lightning Web Components (LWC), Aura components, and Visualforce pages. Covers SLDS 2 compliance, accessibility (WCAG 2.1 AA), data access pattern selection, component communication rules, XSS prevention, CSRF enforcement, FLS/CRUD in AuraEnabled methods, view state management, and Jest test requirements. Use this skill when building or reviewing any Salesforce UI component to enforce platform-specific security and quality standards. | None | +| [salesforce-flow-design](../skills/salesforce-flow-design/SKILL.md) | Salesforce Flow architecture decisions, flow type selection, bulk safety validation, and fault handling standards. Use this skill when designing or reviewing Record-Triggered, Screen, Autolaunched, Scheduled, or Platform Event flows to ensure correct type selection, no DML/Get Records in loops, proper fault connectors on all data-changing elements, and appropriate automation density checks before deployment. | None | | [sandbox-npm-install](../skills/sandbox-npm-install/SKILL.md) | Install npm packages in a Docker sandbox environment. Use this skill whenever you need to install, reinstall, or update node_modules inside a container where the workspace is mounted via virtiofs. Native binaries (esbuild, lightningcss, rollup) crash on virtiofs, so packages must be installed on the local ext4 filesystem and symlinked back. | `scripts/install.sh` | | [scaffolding-oracle-to-postgres-migration-test-project](../skills/scaffolding-oracle-to-postgres-migration-test-project/SKILL.md) | Scaffolds an xUnit integration test project for validating Oracle-to-PostgreSQL database migration behavior in .NET solutions. Creates the test project, transaction-rollback base class, and seed data manager. Use when setting up test infrastructure before writing migration integration tests, or when a test project is needed for Oracle-to-PostgreSQL validation. | None | | [scoutqa-test](../skills/scoutqa-test/SKILL.md) | This skill should be used when the user asks to "test this website", "run exploratory testing", "check for accessibility issues", "verify the login flow works", "find bugs on this page", or requests automated QA testing. Triggers on web application testing scenarios including smoke tests, accessibility audits, e-commerce flows, and user flow validation using ScoutQA CLI. Use this skill proactively after implementing web application features to verify they work correctly. | None | diff --git a/plugins/salesforce-development/.github/plugin/plugin.json b/plugins/salesforce-development/.github/plugin/plugin.json new file mode 100644 index 00000000..f29a9c10 --- /dev/null +++ b/plugins/salesforce-development/.github/plugin/plugin.json @@ -0,0 +1,32 @@ +{ + "name": "salesforce-development", + "description": "Complete Salesforce agentic development environment covering Apex & Triggers, Flow automation, Lightning Web Components, Aura components, and Visualforce pages.", + "version": "1.1.0", + "author": { + "name": "TemitayoAfolabi" + }, + "repository": "https://github.com/github/awesome-copilot", + "license": "MIT", + "keywords": [ + "salesforce", + "apex", + "triggers", + "lwc", + "aura", + "flow", + "visualforce", + "crm", + "salesforce-dx" + ], + "agents": [ + "./agents/salesforce-apex-triggers.md", + "./agents/salesforce-aura-lwc.md", + "./agents/salesforce-flow.md", + "./agents/salesforce-visualforce.md" + ], + "skills": [ + "./skills/salesforce-apex-quality/", + "./skills/salesforce-flow-design/", + "./skills/salesforce-component-standards/" + ] +} diff --git a/plugins/salesforce-development/README.md b/plugins/salesforce-development/README.md new file mode 100644 index 00000000..9eea6e16 --- /dev/null +++ b/plugins/salesforce-development/README.md @@ -0,0 +1,37 @@ +# Salesforce Development Plugin + +Complete Salesforce agentic development environment covering Apex & Triggers, Flow automation, Lightning Web Components (LWC), Aura components, and Visualforce pages. + +## Installation + +```bash +copilot plugin install salesforce-development@awesome-copilot +``` + +## What's Included + +### Agents + +| Agent | Description | +|-------|-------------| +| `salesforce-apex-triggers` | Implement Salesforce business logic using Apex classes and triggers with production-quality code following Salesforce best practices. | +| `salesforce-aura-lwc` | Implement Salesforce UI components using Lightning Web Components and Aura components following Lightning framework best practices. | +| `salesforce-flow` | Implement business automation using Salesforce Flow following declarative automation best practices. | +| `salesforce-visualforce` | Implement Visualforce pages and controllers following Salesforce MVC architecture and best practices. | + +## Usage + +Once installed, switch to any of the Salesforce agents in GitHub Copilot Chat depending on what you are building: + +- Use **`salesforce-apex-triggers`** for backend business logic, trigger handlers, utility classes, and test coverage +- Use **`salesforce-aura-lwc`** for building Lightning Web Components or Aura component UI +- Use **`salesforce-flow`** for declarative automation including Record-Triggered, Screen, Autolaunched, and Scheduled flows +- Use **`salesforce-visualforce`** for Visualforce pages and their Apex controllers + +## Source + +This plugin is part of [Awesome Copilot](https://github.com/github/awesome-copilot), a community-driven collection of GitHub Copilot extensions. + +## License + +MIT diff --git a/skills/salesforce-apex-quality/SKILL.md b/skills/salesforce-apex-quality/SKILL.md new file mode 100644 index 00000000..cfe0f774 --- /dev/null +++ b/skills/salesforce-apex-quality/SKILL.md @@ -0,0 +1,158 @@ +--- +name: salesforce-apex-quality +description: 'Apex code quality guardrails for Salesforce development. Enforces bulk-safety rules (no SOQL/DML in loops), sharing model requirements, CRUD/FLS security, SOQL injection prevention, PNB test coverage (Positive / Negative / Bulk), and modern Apex idioms. Use this skill when reviewing or generating Apex classes, trigger handlers, batch jobs, or test classes to catch governor limit risks, security gaps, and quality issues before deployment.' +--- + +# Salesforce Apex Quality Guardrails + +Apply these checks to every Apex class, trigger, and test file you write or review. + +## Step 1 β€” Governor Limit Safety Check + +Scan for these patterns before declaring any Apex file acceptable: + +### SOQL and DML in Loops β€” Automatic Fail + +```apex +// ❌ NEVER β€” causes LimitException at scale +for (Account a : accounts) { + List contacts = [SELECT Id FROM Contact WHERE AccountId = :a.Id]; // SOQL in loop + update a; // DML in loop +} + +// βœ… ALWAYS β€” collect, then query/update once +Set accountIds = new Map(accounts).keySet(); +Map> contactsByAccount = new Map>(); +for (Contact c : [SELECT Id, AccountId FROM Contact WHERE AccountId IN :accountIds]) { + if (!contactsByAccount.containsKey(c.AccountId)) { + contactsByAccount.put(c.AccountId, new List()); + } + contactsByAccount.get(c.AccountId).add(c); +} +update accounts; // DML once, outside the loop +``` + +Rule: if you see `[SELECT` or `Database.query`, `insert`, `update`, `delete`, `upsert`, `merge` inside a `for` loop body β€” stop and refactor before proceeding. + +## Step 2 β€” Sharing Model Verification + +Every class must declare its sharing intent explicitly. Undeclared sharing inherits from the caller β€” unpredictable behaviour. + +| Declaration | When to use | +|---|---| +| `public with sharing class Foo` | Default for all service, handler, selector, and controller classes | +| `public without sharing class Foo` | Only when the class must run elevated (e.g. system-level logging, trigger bypass). Requires a code comment explaining why. | +| `public inherited sharing class Foo` | Framework entry points that should respect the caller's sharing context | + +If a class does not have one of these three declarations, **add it before writing anything else**. + +## Step 3 β€” CRUD / FLS Enforcement + +Apex code that reads or writes records on behalf of a user must verify object and field access. The platform does **not** enforce FLS or CRUD automatically in Apex. + +```apex +// Check before querying a field +if (!Schema.sObjectType.Contact.fields.Email.isAccessible()) { + throw new System.NoAccessException(); +} + +// Or use WITH USER_MODE in SOQL (API 56.0+) +List contacts = [SELECT Id, Email FROM Contact WHERE AccountId = :accId WITH USER_MODE]; + +// Or use Database.query with AccessLevel +List contacts = Database.query('SELECT Id, Email FROM Contact', AccessLevel.USER_MODE); +``` + +Rule: any Apex method callable from a UI component, REST endpoint, or `@InvocableMethod` **must** enforce CRUD/FLS. Internal service methods called only from trusted contexts may use `with sharing` instead. + +## Step 4 β€” SOQL Injection Prevention + +```apex +// ❌ NEVER β€” concatenates user input into SOQL string +String soql = 'SELECT Id FROM Account WHERE Name = \'' + userInput + '\''; + +// βœ… ALWAYS β€” bind variable +String soql = [SELECT Id FROM Account WHERE Name = :userInput]; + +// βœ… For dynamic SOQL with user-controlled field names β€” validate against a whitelist +Set allowedFields = new Set{'Name', 'Industry', 'AnnualRevenue'}; +if (!allowedFields.contains(userInput)) { + throw new IllegalArgumentException('Field not permitted: ' + userInput); +} +``` + +## Step 5 β€” Modern Apex Idioms + +Prefer current language features (API 62.0 / Winter '25+): + +| Old pattern | Modern replacement | +|---|---| +| `if (obj != null) { x = obj.Field__c; }` | `x = obj?.Field__c;` | +| `x = (y != null) ? y : defaultVal;` | `x = y ?? defaultVal;` | +| `System.assertEquals(expected, actual)` | `Assert.areEqual(expected, actual)` | +| `System.assert(condition)` | `Assert.isTrue(condition)` | +| `[SELECT ... WHERE ...]` with no sharing context | `[SELECT ... WHERE ... WITH USER_MODE]` | + +## Step 6 β€” PNB Test Coverage Checklist + +Every feature must be tested across all three paths. Missing any one of these is a quality failure: + +### Positive Path +- Expected input β†’ expected output. +- Assert the exact field values, record counts, or return values β€” not just that no exception was thrown. + +### Negative Path +- Invalid input, null values, empty collections, and error conditions. +- Assert that exceptions are thrown with the correct type and message. +- Assert that no records were mutated when the operation should have failed cleanly. + +### Bulk Path +- Insert/update/delete **200–251 records** in a single test transaction. +- Assert that all records processed correctly β€” no partial failures from governor limits. +- Use `Test.startTest()` / `Test.stopTest()` to isolate governor limit counters for async work. + +### Test Class Rules +```apex +@isTest(SeeAllData=false) // Required β€” no exceptions without a documented reason +private class AccountServiceTest { + + @TestSetup + static void makeData() { + // Create all test data here β€” use a factory if one exists in the project + } + + @isTest + static void givenValidInput_whenProcessAccounts_thenFieldsUpdated() { + // Positive path + List accounts = [SELECT Id FROM Account LIMIT 10]; + Test.startTest(); + AccountService.processAccounts(accounts); + Test.stopTest(); + // Assert meaningful outcomes β€” not just no exception + List updated = [SELECT Status__c FROM Account WHERE Id IN :accounts]; + Assert.areEqual('Processed', updated[0].Status__c, 'Status should be Processed'); + } +} +``` + +## Step 7 β€” Trigger Architecture Checklist + +- [ ] One trigger per object. If a second trigger exists, consolidate into the handler. +- [ ] Trigger body contains only: context checks, handler invocation, and routing logic. +- [ ] No business logic, SOQL, or DML directly in the trigger body. +- [ ] If a trigger framework (Trigger Actions Framework, ff-apex-common, custom base class) is already in use β€” extend it. Do not create a parallel pattern. +- [ ] Handler class is `with sharing` unless the trigger requires elevated access. + +## Quick Reference β€” Hardcoded Anti-Patterns Summary + +| Pattern | Action | +|---|---| +| SOQL inside `for` loop | Refactor: query before the loop, operate on collections | +| DML inside `for` loop | Refactor: collect mutations, DML once after the loop | +| Class missing sharing declaration | Add `with sharing` (or document why `without sharing`) | +| `escape="false"` on user data (VF) | Remove β€” auto-escaping enforces XSS prevention | +| Empty `catch` block | Add logging and appropriate re-throw or error handling | +| String-concatenated SOQL with user input | Replace with bind variable or whitelist validation | +| Test with no assertion | Add a meaningful `Assert.*` call | +| `System.assert` / `System.assertEquals` style | Upgrade to `Assert.isTrue` / `Assert.areEqual` | +| Hardcoded record ID (`'001...'`) | Replace with queried or inserted test record ID | diff --git a/skills/salesforce-component-standards/SKILL.md b/skills/salesforce-component-standards/SKILL.md new file mode 100644 index 00000000..87fd9c76 --- /dev/null +++ b/skills/salesforce-component-standards/SKILL.md @@ -0,0 +1,182 @@ +--- +name: salesforce-component-standards +description: 'Quality standards for Salesforce Lightning Web Components (LWC), Aura components, and Visualforce pages. Covers SLDS 2 compliance, accessibility (WCAG 2.1 AA), data access pattern selection, component communication rules, XSS prevention, CSRF enforcement, FLS/CRUD in AuraEnabled methods, view state management, and Jest test requirements. Use this skill when building or reviewing any Salesforce UI component to enforce platform-specific security and quality standards.' +--- + +# Salesforce Component Quality Standards + +Apply these checks to every LWC, Aura component, and Visualforce page you write or review. + +## Section 1 β€” LWC Quality Standards + +### 1.1 Data Access Pattern Selection + +Choose the right data access pattern before writing JavaScript controller code: + +| Use case | Pattern | Why | +|---|---|---| +| Read a single record reactively (follows navigation) | `@wire(getRecord, { recordId, fields })` | Lightning Data Service β€” cached, reactive | +| Standard CRUD form for a single object | `` or `` | Built-in FLS, CRUD, and accessibility | +| Complex server query or filtered list | `@wire(apexMethodName, { param })` on a `cacheable=true` method | Allows caching; wire re-fires on param change | +| User-triggered action, DML, or non-cacheable server call | Imperative `apexMethodName(params).then(...).catch(...)` | Required for DML β€” wired methods cannot be `@AuraEnabled` without `cacheable=true` | +| Cross-component communication (no shared parent) | Lightning Message Service (LMS) | Decoupled, works across DOM boundaries | +| Multi-object graph relationships | GraphQL `@wire(gql, { query, variables })` | Single round-trip for complex related data | + +### 1.2 Security Rules + +| Rule | Enforcement | +|---|---| +| No raw user data in `innerHTML` | Use `{expression}` binding in the template β€” the framework auto-escapes. Never use `this.template.querySelector('.el').innerHTML = userValue` | +| Apex `@AuraEnabled` methods enforce CRUD/FLS | Use `WITH USER_MODE` in SOQL or explicit `Schema.sObjectType` checks | +| No hardcoded org-specific IDs in component JavaScript | Query or pass as a prop β€” never embed record IDs in source | +| `@api` properties from parent: validate before use | A parent can pass anything β€” validate type and range before using as a query parameter | + +### 1.3 SLDS 2 and Styling Standards + +- **Never** hardcode colours: `color: #FF3366` β†’ use `color: var(--slds-c-button-brand-color-background)` or a semantic SLDS token. +- **Never** override SLDS classes with `!important` β€” compose with custom CSS properties. +- Use `` base components wherever they exist: `lightning-button`, `lightning-input`, `lightning-datatable`, `lightning-card`, etc. +- Base components include built-in SLDS 2, dark mode, and accessibility β€” avoid reimplementing their behaviour. +- If using custom CSS, test in both **light mode** and **dark mode** before declaring done. + +### 1.4 Accessibility Requirements (WCAG 2.1 AA) + +Every LWC component must pass all of these before it is considered done: + +- [ ] All form inputs have `