mirror of
https://github.com/github/awesome-copilot.git
synced 2026-04-11 02:35:55 +00:00
feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain) (#1248)
* feat: add 3 agent security skills (MCP audit, OWASP compliance, supply chain) - mcp-security-audit: Audit .mcp.json files for hardcoded secrets, shell injection, unpinned versions, dangerous command patterns - agent-owasp-compliance: Check agent systems against OWASP ASI 2026 Top 10 risks with compliance report generation - agent-supply-chain: SHA-256 integrity manifests, tamper detection, version pinning audit, promotion gates for agent plugins Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address all 9 review comments 1. Added 3 new skills to docs/README.skills.md index 2. Added imports (json, re) to shell injection check snippet 3. Updated unpinned deps wording to match code behavior (@latest only) 4. Moved check_secrets() outside per-server loop to avoid duplicates 5. Added imports note to verify_manifest snippet 6. Updated promotion_check to support both .github/plugin and .claude-plugin layouts 7. Updated CI example to cd into plugin directory before verifying 8. Added check sections for all 10 ASI controls (was missing 03, 04, 06, 08, 10) 9. Made ASI-01 code snippet runnable with actual file scanning implementation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: regenerate docs/README.skills.md via npm start Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Imran Siddique
GitHub
323
skills/agent-owasp-compliance/SKILL.md
Normal file
323
skills/agent-owasp-compliance/SKILL.md
Normal file
@@ -0,0 +1,323 @@
|
||||
---
|
||||
name: agent-owasp-compliance
|
||||
description: |
|
||||
Check any AI agent codebase against the OWASP Agentic Security Initiative (ASI) Top 10 risks.
|
||||
Use this skill when:
|
||||
- Evaluating an agent system's security posture before production deployment
|
||||
- Running a compliance check against OWASP ASI 2026 standards
|
||||
- Mapping existing security controls to the 10 agentic risks
|
||||
- Generating a compliance report for security review or audit
|
||||
- Comparing agent framework security features against the standard
|
||||
- Any request like "is my agent OWASP compliant?", "check ASI compliance", or "agentic security audit"
|
||||
---
|
||||
|
||||
# Agent OWASP ASI Compliance Check
|
||||
|
||||
Evaluate AI agent systems against the OWASP Agentic Security Initiative (ASI) Top 10 — the industry standard for agent security posture.
|
||||
|
||||
## Overview
|
||||
|
||||
The OWASP ASI Top 10 defines the critical security risks specific to autonomous AI agents — not LLMs, not chatbots, but agents that call tools, access systems, and act on behalf of users. This skill checks whether your agent implementation addresses each risk.
|
||||
|
||||
```
|
||||
Codebase → Scan for each ASI control:
|
||||
ASI-01: Prompt Injection Protection
|
||||
ASI-02: Tool Use Governance
|
||||
ASI-03: Agency Boundaries
|
||||
ASI-04: Escalation Controls
|
||||
ASI-05: Trust Boundary Enforcement
|
||||
ASI-06: Logging & Audit
|
||||
ASI-07: Identity Management
|
||||
ASI-08: Policy Integrity
|
||||
ASI-09: Supply Chain Verification
|
||||
ASI-10: Behavioral Monitoring
|
||||
→ Generate Compliance Report (X/10 covered)
|
||||
```
|
||||
|
||||
## The 10 Risks
|
||||
|
||||
| Risk | Name | What to Look For |
|
||||
|------|------|-----------------|
|
||||
| ASI-01 | Prompt Injection | Input validation before tool calls, not just LLM output filtering |
|
||||
| ASI-02 | Insecure Tool Use | Tool allowlists, argument validation, no raw shell execution |
|
||||
| ASI-03 | Excessive Agency | Capability boundaries, scope limits, principle of least privilege |
|
||||
| ASI-04 | Unauthorized Escalation | Privilege checks before sensitive operations, no self-promotion |
|
||||
| ASI-05 | Trust Boundary Violation | Trust verification between agents, signed credentials, no blind trust |
|
||||
| ASI-06 | Insufficient Logging | Structured audit trail for all tool calls, tamper-evident logs |
|
||||
| ASI-07 | Insecure Identity | Cryptographic agent identity, not just string names |
|
||||
| ASI-08 | Policy Bypass | Deterministic policy enforcement, no LLM-based permission checks |
|
||||
| ASI-09 | Supply Chain Integrity | Signed plugins/tools, integrity verification, dependency auditing |
|
||||
| ASI-10 | Behavioral Anomaly | Drift detection, circuit breakers, kill switch capability |
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-01: Prompt Injection Protection
|
||||
|
||||
Look for input validation that runs **before** tool execution, not after LLM generation.
|
||||
|
||||
```python
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
def check_asi_01(project_path: str) -> dict:
|
||||
"""ASI-01: Is user input validated before reaching tool execution?"""
|
||||
positive_patterns = [
|
||||
"input_validation", "validate_input", "sanitize",
|
||||
"classify_intent", "prompt_injection", "threat_detect",
|
||||
"PolicyEvaluator", "PolicyEngine", "check_content",
|
||||
]
|
||||
negative_patterns = [
|
||||
r"eval\(", r"exec\(", r"subprocess\.run\(.*shell=True",
|
||||
r"os\.system\(",
|
||||
]
|
||||
|
||||
# Scan Python files for signals
|
||||
root = Path(project_path)
|
||||
positive_matches = []
|
||||
negative_matches = []
|
||||
|
||||
for py_file in root.rglob("*.py"):
|
||||
content = py_file.read_text(errors="ignore")
|
||||
for pattern in positive_patterns:
|
||||
if pattern in content:
|
||||
positive_matches.append(f"{py_file.name}: {pattern}")
|
||||
for pattern in negative_patterns:
|
||||
if re.search(pattern, content):
|
||||
negative_matches.append(f"{py_file.name}: {pattern}")
|
||||
|
||||
positive_found = len(positive_matches) > 0
|
||||
negative_found = len(negative_matches) > 0
|
||||
|
||||
return {
|
||||
"risk": "ASI-01",
|
||||
"name": "Prompt Injection",
|
||||
"status": "pass" if positive_found and not negative_found else "fail",
|
||||
"controls_found": positive_matches,
|
||||
"vulnerabilities": negative_matches,
|
||||
"recommendation": "Add input validation before tool execution, not just output filtering"
|
||||
}
|
||||
```
|
||||
|
||||
**What passing looks like:**
|
||||
```python
|
||||
# GOOD: Validate before tool execution
|
||||
result = policy_engine.evaluate(user_input)
|
||||
if result.action == "deny":
|
||||
return "Request blocked by policy"
|
||||
tool_result = await execute_tool(validated_input)
|
||||
```
|
||||
|
||||
**What failing looks like:**
|
||||
```python
|
||||
# BAD: User input goes directly to tool
|
||||
tool_result = await execute_tool(user_input) # No validation
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-02: Insecure Tool Use
|
||||
|
||||
Verify tools have allowlists, argument validation, and no unrestricted execution.
|
||||
|
||||
**What to search for:**
|
||||
- Tool registration with explicit allowlists (not open-ended)
|
||||
- Argument validation before tool execution
|
||||
- No `subprocess.run(shell=True)` with user-controlled input
|
||||
- No `eval()` or `exec()` on agent-generated code without sandbox
|
||||
|
||||
**Passing example:**
|
||||
```python
|
||||
ALLOWED_TOOLS = {"search", "read_file", "create_ticket"}
|
||||
|
||||
def execute_tool(name: str, args: dict):
|
||||
if name not in ALLOWED_TOOLS:
|
||||
raise PermissionError(f"Tool '{name}' not in allowlist")
|
||||
# validate args...
|
||||
return tools[name](**validated_args)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-03: Excessive Agency
|
||||
|
||||
Verify agent capabilities are bounded — not open-ended.
|
||||
|
||||
**What to search for:**
|
||||
- Explicit capability lists or execution rings
|
||||
- Scope limits on what the agent can access
|
||||
- Principle of least privilege applied to tool access
|
||||
|
||||
**Failing:** Agent has access to all tools by default.
|
||||
**Passing:** Agent capabilities defined as a fixed allowlist, unknown tools denied.
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-04: Unauthorized Escalation
|
||||
|
||||
Verify agents cannot promote their own privileges.
|
||||
|
||||
**What to search for:**
|
||||
- Privilege level checks before sensitive operations
|
||||
- No self-promotion patterns (agent changing its own trust score or role)
|
||||
- Escalation requires external attestation (human or SRE witness)
|
||||
|
||||
**Failing:** Agent can modify its own configuration or permissions.
|
||||
**Passing:** Privilege changes require out-of-band approval (e.g., Ring 0 requires SRE attestation).
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-05: Trust Boundary Violation
|
||||
|
||||
In multi-agent systems, verify that agents verify each other's identity before accepting instructions.
|
||||
|
||||
**What to search for:**
|
||||
- Agent identity verification (DIDs, signed tokens, API keys)
|
||||
- Trust score checks before accepting delegated tasks
|
||||
- No blind trust of inter-agent messages
|
||||
- Delegation narrowing (child scope <= parent scope)
|
||||
|
||||
**Passing example:**
|
||||
```python
|
||||
def accept_task(sender_id: str, task: dict):
|
||||
trust = trust_registry.get_trust(sender_id)
|
||||
if not trust.meets_threshold(0.7):
|
||||
raise PermissionError(f"Agent {sender_id} trust too low: {trust.current()}")
|
||||
if not verify_signature(task, sender_id):
|
||||
raise SecurityError("Task signature verification failed")
|
||||
return process_task(task)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-06: Insufficient Logging
|
||||
|
||||
Verify all agent actions produce structured, tamper-evident audit entries.
|
||||
|
||||
**What to search for:**
|
||||
- Structured logging for every tool call (not just print statements)
|
||||
- Audit entries include: timestamp, agent ID, tool name, args, result, policy decision
|
||||
- Append-only or hash-chained log format
|
||||
- Logs stored separately from agent-writable directories
|
||||
|
||||
**Failing:** Agent actions logged via `print()` or not logged at all.
|
||||
**Passing:** Structured JSONL audit trail with chain hashes, exported to secure storage.
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-07: Insecure Identity
|
||||
|
||||
Verify agents have cryptographic identity, not just string names.
|
||||
|
||||
**Failing indicators:**
|
||||
- Agent identified by `agent_name = "my-agent"` (string only)
|
||||
- No authentication between agents
|
||||
- Shared credentials across agents
|
||||
|
||||
**Passing indicators:**
|
||||
- DID-based identity (`did:web:`, `did:key:`)
|
||||
- Ed25519 or similar cryptographic signing
|
||||
- Per-agent credentials with rotation
|
||||
- Identity bound to specific capabilities
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-08: Policy Bypass
|
||||
|
||||
Verify policy enforcement is deterministic — not LLM-based.
|
||||
|
||||
**What to search for:**
|
||||
- Policy evaluation uses deterministic logic (YAML rules, code predicates)
|
||||
- No LLM calls in the enforcement path
|
||||
- Policy checks cannot be skipped or overridden by the agent
|
||||
- Fail-closed behavior (if policy check errors, action is denied)
|
||||
|
||||
**Failing:** Agent decides its own permissions via prompt ("Am I allowed to...?").
|
||||
**Passing:** PolicyEvaluator.evaluate() returns allow/deny in <0.1ms, no LLM involved.
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-09: Supply Chain Integrity
|
||||
|
||||
Verify agent plugins and tools have integrity verification.
|
||||
|
||||
**What to search for:**
|
||||
- `INTEGRITY.json` or manifest files with SHA-256 hashes
|
||||
- Signature verification on plugin installation
|
||||
- Dependency pinning (no `@latest`, `>=` without upper bound)
|
||||
- SBOM generation
|
||||
|
||||
---
|
||||
|
||||
## Check ASI-10: Behavioral Anomaly
|
||||
|
||||
Verify the system can detect and respond to agent behavioral drift.
|
||||
|
||||
**What to search for:**
|
||||
- Circuit breakers that trip on repeated failures
|
||||
- Trust score decay over time (temporal decay)
|
||||
- Kill switch or emergency stop capability
|
||||
- Anomaly detection on tool call patterns (frequency, targets, timing)
|
||||
|
||||
**Failing:** No mechanism to stop a misbehaving agent automatically.
|
||||
**Passing:** Circuit breaker trips after N failures, trust decays without activity, kill switch available.
|
||||
|
||||
---
|
||||
|
||||
## Compliance Report Format
|
||||
|
||||
```markdown
|
||||
# OWASP ASI Compliance Report
|
||||
Generated: 2026-04-01
|
||||
Project: my-agent-system
|
||||
|
||||
## Summary: 7/10 Controls Covered
|
||||
|
||||
| Risk | Status | Finding |
|
||||
|------|--------|---------|
|
||||
| ASI-01 Prompt Injection | PASS | PolicyEngine validates input before tool calls |
|
||||
| ASI-02 Insecure Tool Use | PASS | Tool allowlist enforced in governance.py |
|
||||
| ASI-03 Excessive Agency | PASS | Execution rings limit capabilities |
|
||||
| ASI-04 Unauthorized Escalation | PASS | Ring promotion requires attestation |
|
||||
| ASI-05 Trust Boundary | FAIL | No identity verification between agents |
|
||||
| ASI-06 Insufficient Logging | PASS | AuditChain with SHA-256 chain hashes |
|
||||
| ASI-07 Insecure Identity | FAIL | Agents use string names, no crypto identity |
|
||||
| ASI-08 Policy Bypass | PASS | Deterministic PolicyEvaluator, no LLM in path |
|
||||
| ASI-09 Supply Chain | FAIL | No integrity manifests or plugin signing |
|
||||
| ASI-10 Behavioral Anomaly | PASS | Circuit breakers and trust decay active |
|
||||
|
||||
## Critical Gaps
|
||||
- ASI-05: Add agent identity verification using DIDs or signed tokens
|
||||
- ASI-07: Replace string agent names with cryptographic identity
|
||||
- ASI-09: Generate INTEGRITY.json manifests for all plugins
|
||||
|
||||
## Recommendation
|
||||
Install agent-governance-toolkit for reference implementations of all 10 controls:
|
||||
pip install agent-governance-toolkit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Assessment Questions
|
||||
|
||||
Use these to rapidly assess an agent system:
|
||||
|
||||
1. **Does user input pass through validation before reaching any tool?** (ASI-01)
|
||||
2. **Is there an explicit list of what tools the agent can call?** (ASI-02)
|
||||
3. **Can the agent do anything, or are its capabilities bounded?** (ASI-03)
|
||||
4. **Can the agent promote its own privileges?** (ASI-04)
|
||||
5. **Do agents verify each other's identity before accepting tasks?** (ASI-05)
|
||||
6. **Is every tool call logged with enough detail to replay it?** (ASI-06)
|
||||
7. **Does each agent have a unique cryptographic identity?** (ASI-07)
|
||||
8. **Is policy enforcement deterministic (not LLM-based)?** (ASI-08)
|
||||
9. **Are plugins/tools integrity-verified before use?** (ASI-09)
|
||||
10. **Is there a circuit breaker or kill switch?** (ASI-10)
|
||||
|
||||
If you answer "no" to any of these, that's a gap to address.
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [OWASP Agentic AI Threats](https://owasp.org/www-project-agentic-ai-threats/)
|
||||
- [Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) — Reference implementation covering 10/10 ASI controls
|
||||
- [agent-governance skill](https://github.com/github/awesome-copilot/tree/main/skills/agent-governance) — Governance patterns for agent systems
|
||||
339
skills/agent-supply-chain/SKILL.md
Normal file
339
skills/agent-supply-chain/SKILL.md
Normal file
@@ -0,0 +1,339 @@
|
||||
---
|
||||
name: agent-supply-chain
|
||||
description: |
|
||||
Verify supply chain integrity for AI agent plugins, tools, and dependencies. Use this skill when:
|
||||
- Generating SHA-256 integrity manifests for agent plugins or tool packages
|
||||
- Verifying that installed plugins match their published manifests
|
||||
- Detecting tampered, modified, or untracked files in agent tool directories
|
||||
- Auditing dependency pinning and version policies for agent components
|
||||
- Building provenance chains for agent plugin promotion (dev → staging → production)
|
||||
- Any request like "verify plugin integrity", "generate manifest", "check supply chain", or "sign this plugin"
|
||||
---
|
||||
|
||||
# Agent Supply Chain Integrity
|
||||
|
||||
Generate and verify integrity manifests for AI agent plugins and tools. Detect tampering, enforce version pinning, and establish supply chain provenance.
|
||||
|
||||
## Overview
|
||||
|
||||
Agent plugins and MCP servers have the same supply chain risks as npm packages or container images — except the ecosystem has no equivalent of npm provenance, Sigstore, or SLSA. This skill fills that gap.
|
||||
|
||||
```
|
||||
Plugin Directory → Hash All Files (SHA-256) → Generate INTEGRITY.json
|
||||
↓
|
||||
Later: Plugin Directory → Re-Hash Files → Compare Against INTEGRITY.json
|
||||
↓
|
||||
Match? VERIFIED : TAMPERED
|
||||
```
|
||||
|
||||
## When to Use
|
||||
|
||||
- Before promoting a plugin from development to production
|
||||
- During code review of plugin PRs
|
||||
- As a CI step to verify no files were modified after review
|
||||
- When auditing third-party agent tools or MCP servers
|
||||
- Building a plugin marketplace with integrity requirements
|
||||
|
||||
---
|
||||
|
||||
## Pattern 1: Generate Integrity Manifest
|
||||
|
||||
Create a deterministic `INTEGRITY.json` with SHA-256 hashes of all plugin files.
|
||||
|
||||
```python
|
||||
import hashlib
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
EXCLUDE_DIRS = {".git", "__pycache__", "node_modules", ".venv", ".pytest_cache"}
|
||||
EXCLUDE_FILES = {".DS_Store", "Thumbs.db", "INTEGRITY.json"}
|
||||
|
||||
def hash_file(path: Path) -> str:
|
||||
"""Compute SHA-256 hex digest of a file."""
|
||||
h = hashlib.sha256()
|
||||
with open(path, "rb") as f:
|
||||
for chunk in iter(lambda: f.read(8192), b""):
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
def generate_manifest(plugin_dir: str) -> dict:
|
||||
"""Generate an integrity manifest for a plugin directory."""
|
||||
root = Path(plugin_dir)
|
||||
files = {}
|
||||
|
||||
for path in sorted(root.rglob("*")):
|
||||
if not path.is_file():
|
||||
continue
|
||||
if path.name in EXCLUDE_FILES:
|
||||
continue
|
||||
if any(part in EXCLUDE_DIRS for part in path.relative_to(root).parts):
|
||||
continue
|
||||
rel = path.relative_to(root).as_posix()
|
||||
files[rel] = hash_file(path)
|
||||
|
||||
# Chain hash: SHA-256 of all file hashes concatenated in sorted order
|
||||
chain = hashlib.sha256()
|
||||
for key in sorted(files.keys()):
|
||||
chain.update(files[key].encode("ascii"))
|
||||
|
||||
manifest = {
|
||||
"plugin_name": root.name,
|
||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||
"algorithm": "sha256",
|
||||
"file_count": len(files),
|
||||
"files": files,
|
||||
"manifest_hash": chain.hexdigest(),
|
||||
}
|
||||
return manifest
|
||||
|
||||
# Generate and save
|
||||
manifest = generate_manifest("my-plugin/")
|
||||
Path("my-plugin/INTEGRITY.json").write_text(
|
||||
json.dumps(manifest, indent=2) + "\n"
|
||||
)
|
||||
print(f"Generated manifest: {manifest['file_count']} files, "
|
||||
f"hash: {manifest['manifest_hash'][:16]}...")
|
||||
```
|
||||
|
||||
**Output (`INTEGRITY.json`):**
|
||||
```json
|
||||
{
|
||||
"plugin_name": "my-plugin",
|
||||
"generated_at": "2026-04-01T03:00:00+00:00",
|
||||
"algorithm": "sha256",
|
||||
"file_count": 12,
|
||||
"files": {
|
||||
".claude-plugin/plugin.json": "a1b2c3d4...",
|
||||
"README.md": "e5f6a7b8...",
|
||||
"skills/search/SKILL.md": "c9d0e1f2...",
|
||||
"agency.json": "3a4b5c6d..."
|
||||
},
|
||||
"manifest_hash": "7e8f9a0b1c2d3e4f..."
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pattern 2: Verify Integrity
|
||||
|
||||
Check that current files match the manifest.
|
||||
|
||||
```python
|
||||
# Requires: hash_file() and generate_manifest() from Pattern 1 above
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
def verify_manifest(plugin_dir: str) -> tuple[bool, list[str]]:
|
||||
"""Verify plugin files against INTEGRITY.json."""
|
||||
root = Path(plugin_dir)
|
||||
manifest_path = root / "INTEGRITY.json"
|
||||
|
||||
if not manifest_path.exists():
|
||||
return False, ["INTEGRITY.json not found"]
|
||||
|
||||
manifest = json.loads(manifest_path.read_text())
|
||||
recorded = manifest.get("files", {})
|
||||
errors = []
|
||||
|
||||
# Check recorded files
|
||||
for rel_path, expected_hash in recorded.items():
|
||||
full = root / rel_path
|
||||
if not full.exists():
|
||||
errors.append(f"MISSING: {rel_path}")
|
||||
continue
|
||||
actual = hash_file(full)
|
||||
if actual != expected_hash:
|
||||
errors.append(f"MODIFIED: {rel_path}")
|
||||
|
||||
# Check for new untracked files
|
||||
current = generate_manifest(plugin_dir)
|
||||
for rel_path in current["files"]:
|
||||
if rel_path not in recorded:
|
||||
errors.append(f"UNTRACKED: {rel_path}")
|
||||
|
||||
return len(errors) == 0, errors
|
||||
|
||||
# Verify
|
||||
passed, errors = verify_manifest("my-plugin/")
|
||||
if passed:
|
||||
print("VERIFIED: All files match manifest")
|
||||
else:
|
||||
print(f"FAILED: {len(errors)} issue(s)")
|
||||
for e in errors:
|
||||
print(f" {e}")
|
||||
```
|
||||
|
||||
**Output on tampered plugin:**
|
||||
```
|
||||
FAILED: 3 issue(s)
|
||||
MODIFIED: skills/search/SKILL.md
|
||||
MISSING: agency.json
|
||||
UNTRACKED: backdoor.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pattern 3: Dependency Version Audit
|
||||
|
||||
Check that agent dependencies use pinned versions.
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
def audit_versions(config_path: str) -> list[dict]:
|
||||
"""Audit dependency version pinning in a config file."""
|
||||
findings = []
|
||||
path = Path(config_path)
|
||||
content = path.read_text()
|
||||
|
||||
if path.name == "package.json":
|
||||
data = json.loads(content)
|
||||
for section in ("dependencies", "devDependencies"):
|
||||
for pkg, ver in data.get(section, {}).items():
|
||||
if ver.startswith("^") or ver.startswith("~") or ver == "*" or ver == "latest":
|
||||
findings.append({
|
||||
"package": pkg,
|
||||
"version": ver,
|
||||
"severity": "HIGH" if ver in ("*", "latest") else "MEDIUM",
|
||||
"fix": f'Pin to exact: "{pkg}": "{ver.lstrip("^~")}"'
|
||||
})
|
||||
|
||||
elif path.name in ("requirements.txt", "pyproject.toml"):
|
||||
for line in content.splitlines():
|
||||
line = line.strip()
|
||||
if ">=" in line and "<" not in line:
|
||||
findings.append({
|
||||
"package": line.split(">=")[0].strip(),
|
||||
"version": line,
|
||||
"severity": "MEDIUM",
|
||||
"fix": f"Add upper bound: {line},<next_major"
|
||||
})
|
||||
|
||||
return findings
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Pattern 4: Promotion Gate
|
||||
|
||||
Use integrity verification as a gate before promoting plugins.
|
||||
|
||||
```python
|
||||
def promotion_check(plugin_dir: str) -> dict:
|
||||
"""Check if a plugin is ready for production promotion."""
|
||||
checks = {}
|
||||
|
||||
# 1. Integrity manifest exists and verifies
|
||||
passed, errors = verify_manifest(plugin_dir)
|
||||
checks["integrity"] = {
|
||||
"passed": passed,
|
||||
"errors": errors
|
||||
}
|
||||
|
||||
# 2. Required files exist
|
||||
root = Path(plugin_dir)
|
||||
required = ["README.md"]
|
||||
missing = [f for f in required if not (root / f).exists()]
|
||||
|
||||
# Require at least one plugin manifest (supports both layouts)
|
||||
manifest_paths = [
|
||||
root / ".github/plugin/plugin.json",
|
||||
root / ".claude-plugin/plugin.json",
|
||||
]
|
||||
if not any(p.exists() for p in manifest_paths):
|
||||
missing.append(".github/plugin/plugin.json (or .claude-plugin/plugin.json)")
|
||||
|
||||
checks["required_files"] = {
|
||||
"passed": len(missing) == 0,
|
||||
"missing": missing
|
||||
}
|
||||
|
||||
# 3. No unpinned dependencies
|
||||
mcp_path = root / ".mcp.json"
|
||||
if mcp_path.exists():
|
||||
config = json.loads(mcp_path.read_text())
|
||||
unpinned = []
|
||||
for server in config.get("mcpServers", {}).values():
|
||||
if isinstance(server, dict):
|
||||
for arg in server.get("args", []):
|
||||
if isinstance(arg, str) and "@latest" in arg:
|
||||
unpinned.append(arg)
|
||||
checks["pinned_deps"] = {
|
||||
"passed": len(unpinned) == 0,
|
||||
"unpinned": unpinned
|
||||
}
|
||||
|
||||
# Overall
|
||||
all_passed = all(c["passed"] for c in checks.values())
|
||||
return {"ready": all_passed, "checks": checks}
|
||||
|
||||
result = promotion_check("my-plugin/")
|
||||
if result["ready"]:
|
||||
print("Plugin is ready for production promotion")
|
||||
else:
|
||||
print("Plugin NOT ready:")
|
||||
for name, check in result["checks"].items():
|
||||
if not check["passed"]:
|
||||
print(f" FAILED: {name}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CI Integration
|
||||
|
||||
Add to your GitHub Actions workflow:
|
||||
|
||||
```yaml
|
||||
- name: Verify plugin integrity
|
||||
run: |
|
||||
PLUGIN_DIR="${{ matrix.plugin || '.' }}"
|
||||
cd "$PLUGIN_DIR"
|
||||
python -c "
|
||||
from pathlib import Path
|
||||
import json, hashlib, sys
|
||||
|
||||
def hash_file(p):
|
||||
h = hashlib.sha256()
|
||||
with open(p, 'rb') as f:
|
||||
for c in iter(lambda: f.read(8192), b''):
|
||||
h.update(c)
|
||||
return h.hexdigest()
|
||||
|
||||
manifest = json.loads(Path('INTEGRITY.json').read_text())
|
||||
errors = []
|
||||
for rel, expected in manifest['files'].items():
|
||||
p = Path(rel)
|
||||
if not p.exists():
|
||||
errors.append(f'MISSING: {rel}')
|
||||
elif hash_file(p) != expected:
|
||||
errors.append(f'MODIFIED: {rel}')
|
||||
if errors:
|
||||
for e in errors:
|
||||
print(f'::error::{e}')
|
||||
sys.exit(1)
|
||||
print(f'Verified {len(manifest[\"files\"])} files')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices
|
||||
|
||||
| Practice | Rationale |
|
||||
|----------|-----------|
|
||||
| **Generate manifest after code review** | Ensures reviewed code matches production code |
|
||||
| **Include manifest in the PR** | Reviewers can verify what was hashed |
|
||||
| **Verify in CI before deploy** | Catches post-review modifications |
|
||||
| **Chain hash for tamper evidence** | Single hash represents entire plugin state |
|
||||
| **Exclude build artifacts** | Only hash source files — .git, __pycache__, node_modules excluded |
|
||||
| **Pin all dependency versions** | Unpinned deps = different code on every install |
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [OpenSSF SLSA](https://slsa.dev/) — Supply-chain Levels for Software Artifacts
|
||||
- [npm Provenance](https://docs.npmjs.com/generating-provenance-statements) — Sigstore-based package provenance
|
||||
- [Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) — Includes integrity verification and plugin signing
|
||||
- [OWASP ASI-09: Supply Chain Integrity](https://owasp.org/www-project-agentic-ai-threats/)
|
||||
278
skills/mcp-security-audit/SKILL.md
Normal file
278
skills/mcp-security-audit/SKILL.md
Normal file
@@ -0,0 +1,278 @@
|
||||
---
|
||||
name: mcp-security-audit
|
||||
description: |
|
||||
Audit MCP (Model Context Protocol) server configurations for security issues. Use this skill when:
|
||||
- Reviewing .mcp.json files for security risks
|
||||
- Checking MCP server args for hardcoded secrets or shell injection patterns
|
||||
- Validating that MCP servers use pinned versions (not @latest)
|
||||
- Detecting unpinned dependencies in MCP server configurations
|
||||
- Auditing which MCP servers a project registers and whether they're on an approved list
|
||||
- Checking for environment variable usage vs. hardcoded credentials in MCP configs
|
||||
- Any request like "is my MCP config secure?", "audit my MCP servers", or "check .mcp.json"
|
||||
keywords: [mcp, security, audit, secrets, shell-injection, supply-chain, governance]
|
||||
---
|
||||
|
||||
# MCP Security Audit
|
||||
|
||||
Audit MCP server configurations for security issues — secrets exposure, shell injection, unpinned dependencies, and unapproved servers.
|
||||
|
||||
## Overview
|
||||
|
||||
MCP servers give agents direct tool access to external systems. A misconfigured `.mcp.json` can expose credentials, allow shell injection, or connect to untrusted servers. This skill catches those issues before they reach production.
|
||||
|
||||
```
|
||||
.mcp.json → Parse Servers → Check Each Server:
|
||||
1. Secrets in args/env?
|
||||
2. Shell injection patterns?
|
||||
3. Unpinned versions (@latest)?
|
||||
4. Dangerous commands (eval, bash -c)?
|
||||
5. Server on approved list?
|
||||
→ Generate Report
|
||||
```
|
||||
|
||||
## When to Use
|
||||
|
||||
- Reviewing any `.mcp.json` file in a project
|
||||
- Onboarding a new MCP server to a project
|
||||
- Auditing all MCP servers in a monorepo or plugin marketplace
|
||||
- Pre-commit checks for MCP configuration changes
|
||||
- Security review of agent tool configurations
|
||||
|
||||
---
|
||||
|
||||
## Audit Check 1: Hardcoded Secrets
|
||||
|
||||
Scan MCP server args and env values for hardcoded credentials.
|
||||
|
||||
```python
|
||||
import json
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
SECRET_PATTERNS = [
|
||||
(r'(?i)(api[_-]?key|token|secret|password|credential)\s*[:=]\s*["\'][^"\']{8,}', "Hardcoded secret"),
|
||||
(r'(?i)Bearer\s+[A-Za-z0-9\-._~+/]+=*', "Hardcoded bearer token"),
|
||||
(r'(?i)(ghp_|gho_|ghu_|ghs_|ghr_)[A-Za-z0-9]{30,}', "GitHub token"),
|
||||
(r'sk-[A-Za-z0-9]{20,}', "OpenAI API key"),
|
||||
(r'AKIA[0-9A-Z]{16}', "AWS access key"),
|
||||
(r'-----BEGIN\s+(RSA\s+)?PRIVATE\s+KEY-----', "Private key"),
|
||||
]
|
||||
|
||||
def check_secrets(mcp_config: dict) -> list[dict]:
|
||||
"""Check for hardcoded secrets in MCP server configurations."""
|
||||
findings = []
|
||||
raw = json.dumps(mcp_config)
|
||||
for pattern, description in SECRET_PATTERNS:
|
||||
matches = re.findall(pattern, raw)
|
||||
if matches:
|
||||
findings.append({
|
||||
"severity": "CRITICAL",
|
||||
"check": "hardcoded-secret",
|
||||
"message": f"{description} found in MCP configuration",
|
||||
"evidence": f"Pattern matched: {pattern}",
|
||||
"fix": "Use environment variable references: ${ENV_VAR_NAME}"
|
||||
})
|
||||
return findings
|
||||
```
|
||||
|
||||
**Good practice — use env var references:**
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"my-server": {
|
||||
"command": "node",
|
||||
"args": ["server.js"],
|
||||
"env": {
|
||||
"API_KEY": "${MY_API_KEY}",
|
||||
"DB_URL": "${DATABASE_URL}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Bad — hardcoded credentials:**
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"my-server": {
|
||||
"command": "node",
|
||||
"args": ["server.js", "--api-key", "sk-abc123realkey456"],
|
||||
"env": {
|
||||
"DB_URL": "postgresql://admin:password123@prod-db:5432/main"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Audit Check 2: Shell Injection Patterns
|
||||
|
||||
Detect dangerous command patterns in MCP server args.
|
||||
|
||||
```python
|
||||
import json
|
||||
import re
|
||||
|
||||
DANGEROUS_PATTERNS = [
|
||||
(r'\$\(', "Command substitution $(...)"),
|
||||
(r'`[^`]+`', "Backtick command substitution"),
|
||||
(r';\s*\w', "Command chaining with semicolon"),
|
||||
(r'\|\s*\w', "Pipe to another command"),
|
||||
(r'&&\s*\w', "Command chaining with &&"),
|
||||
(r'\|\|\s*\w', "Command chaining with ||"),
|
||||
(r'(?i)eval\s', "eval usage"),
|
||||
(r'(?i)bash\s+-c\s', "bash -c execution"),
|
||||
(r'(?i)sh\s+-c\s', "sh -c execution"),
|
||||
(r'>\s*/dev/tcp/', "TCP redirect (reverse shell pattern)"),
|
||||
(r'curl\s+.*\|\s*(ba)?sh', "curl pipe to shell"),
|
||||
]
|
||||
|
||||
def check_shell_injection(server_config: dict) -> list[dict]:
|
||||
"""Check MCP server args for shell injection risks."""
|
||||
findings = []
|
||||
args_text = json.dumps(server_config.get("args", []))
|
||||
for pattern, description in DANGEROUS_PATTERNS:
|
||||
if re.search(pattern, args_text):
|
||||
findings.append({
|
||||
"severity": "HIGH",
|
||||
"check": "shell-injection",
|
||||
"message": f"Dangerous pattern in MCP server args: {description}",
|
||||
"fix": "Use direct command execution, not shell interpolation"
|
||||
})
|
||||
return findings
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Audit Check 3: Unpinned Dependencies
|
||||
|
||||
Flag MCP servers using `@latest` in their package references.
|
||||
|
||||
```python
|
||||
def check_pinned_versions(server_config: dict) -> list[dict]:
|
||||
"""Check that MCP server dependencies use pinned versions, not @latest."""
|
||||
findings = []
|
||||
args = server_config.get("args", [])
|
||||
for arg in args:
|
||||
if isinstance(arg, str):
|
||||
if "@latest" in arg:
|
||||
findings.append({
|
||||
"severity": "MEDIUM",
|
||||
"check": "unpinned-dependency",
|
||||
"message": f"Unpinned dependency: {arg}",
|
||||
"fix": f"Pin to specific version: {arg.replace('@latest', '@1.2.3')}"
|
||||
})
|
||||
# npx with unversioned package
|
||||
if arg.startswith("-y") or (not "@" in arg and not arg.startswith("-")):
|
||||
pass # npx flag or plain arg, ok
|
||||
# Check if using npx without -y (interactive prompt in CI)
|
||||
command = server_config.get("command", "")
|
||||
if command == "npx" and "-y" not in args:
|
||||
findings.append({
|
||||
"severity": "LOW",
|
||||
"check": "npx-interactive",
|
||||
"message": "npx without -y flag may prompt interactively in CI",
|
||||
"fix": "Add -y flag: npx -y package-name"
|
||||
})
|
||||
return findings
|
||||
```
|
||||
|
||||
**Good — pinned version:**
|
||||
```json
|
||||
{ "args": ["-y", "my-mcp-server@2.1.0"] }
|
||||
```
|
||||
|
||||
**Bad — unpinned:**
|
||||
```json
|
||||
{ "args": ["-y", "my-mcp-server@latest"] }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Audit Check 4: Full Audit Runner
|
||||
|
||||
Combine all checks into a single audit.
|
||||
|
||||
```python
|
||||
def audit_mcp_config(mcp_path: str) -> dict:
|
||||
"""Run full security audit on an .mcp.json file."""
|
||||
path = Path(mcp_path)
|
||||
if not path.exists():
|
||||
return {"error": f"{mcp_path} not found"}
|
||||
|
||||
config = json.loads(path.read_text(encoding="utf-8"))
|
||||
servers = config.get("mcpServers", {})
|
||||
results = {"file": str(path), "servers": {}, "summary": {}}
|
||||
total_findings = []
|
||||
|
||||
# Run secrets check once on the whole config (not per-server)
|
||||
config_level_findings = check_secrets(config)
|
||||
total_findings.extend(config_level_findings)
|
||||
|
||||
for name, server_config in servers.items():
|
||||
if not isinstance(server_config, dict):
|
||||
continue
|
||||
findings = []
|
||||
findings.extend(check_shell_injection(server_config))
|
||||
findings.extend(check_pinned_versions(server_config))
|
||||
results["servers"][name] = {
|
||||
"command": server_config.get("command", ""),
|
||||
"findings": findings,
|
||||
}
|
||||
total_findings.extend(findings)
|
||||
|
||||
# Summary
|
||||
by_severity = {}
|
||||
for f in total_findings:
|
||||
sev = f["severity"]
|
||||
by_severity[sev] = by_severity.get(sev, 0) + 1
|
||||
|
||||
results["summary"] = {
|
||||
"total_servers": len(servers),
|
||||
"total_findings": len(total_findings),
|
||||
"by_severity": by_severity,
|
||||
"passed": len(total_findings) == 0,
|
||||
}
|
||||
return results
|
||||
```
|
||||
|
||||
**Usage:**
|
||||
```python
|
||||
results = audit_mcp_config(".mcp.json")
|
||||
if not results["summary"]["passed"]:
|
||||
for server, data in results["servers"].items():
|
||||
for finding in data["findings"]:
|
||||
print(f"[{finding['severity']}] {server}: {finding['message']}")
|
||||
print(f" Fix: {finding['fix']}")
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Output Format
|
||||
|
||||
```
|
||||
MCP Security Audit — .mcp.json
|
||||
═══════════════════════════════
|
||||
Servers scanned: 5
|
||||
Findings: 3 (1 CRITICAL, 1 HIGH, 1 MEDIUM)
|
||||
|
||||
[CRITICAL] my-api-server: Hardcoded secret found in MCP configuration
|
||||
Fix: Use environment variable references: ${ENV_VAR_NAME}
|
||||
|
||||
[HIGH] data-processor: Dangerous pattern in MCP server args: bash -c execution
|
||||
Fix: Use direct command execution, not shell interpolation
|
||||
|
||||
[MEDIUM] analytics: Unpinned dependency: analytics-mcp@latest
|
||||
Fix: Pin to specific version: analytics-mcp@2.1.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Related Resources
|
||||
|
||||
- [MCP Specification](https://modelcontextprotocol.io/)
|
||||
- [Agent Governance Toolkit](https://github.com/microsoft/agent-governance-toolkit) — Full governance framework with MCP trust proxy
|
||||
- [OWASP ASI-02: Insecure Tool Use](https://owasp.org/www-project-agentic-ai-threats/)
|
||||
Reference in New Issue
Block a user