mirror of
https://github.com/github/awesome-copilot.git
synced 2026-03-12 04:05:12 +00:00
eb7d223446
* Add doublecheck plugin: three-layer verification pipeline for AI output Adds a new plugin that helps users verify AI-generated content before acting on it. Designed for sensitive contexts (legal, medical, financial, compliance) where hallucinations carry real consequences. Three verification layers: - Self-Audit: extracts verifiable claims, checks internal consistency - Source Verification: web searches per claim, produces URLs for human review - Adversarial Review: assumes errors exist, checks hallucination patterns Supports persistent mode (auto-verifies every factual response inline) and one-shot mode (full report on specific text). Confidence ratings: VERIFIED, PLAUSIBLE, UNVERIFIED, DISPUTED, FABRICATION RISK. Includes: - Skill (skills/doublecheck/) with bundled report template - Agent (agents/doublecheck.agent.md) for interactive verification - Plugin package (plugins/doublecheck/) bundling both Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: fix tools YAML format, remove materialized artifacts - Fix tools frontmatter in agents/doublecheck.agent.md to use standard YAML list format instead of flow sequence with trailing comma - Remove plugins/doublecheck/agents/ and plugins/doublecheck/skills/ from tracking; these paths are in .gitignore as CI-materialized artifacts that should not be committed Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
93 lines
2.7 KiB
Markdown
93 lines
2.7 KiB
Markdown
# Verification Report
|
|
|
|
## Summary
|
|
|
|
**Text verified:** [Brief description of what was checked]
|
|
**Claims extracted:** [N total]
|
|
**Breakdown:**
|
|
|
|
| Rating | Count |
|
|
|--------|-------|
|
|
| VERIFIED | |
|
|
| PLAUSIBLE | |
|
|
| UNVERIFIED | |
|
|
| DISPUTED | |
|
|
| FABRICATION RISK | |
|
|
|
|
**Items requiring attention:** [N items rated DISPUTED or FABRICATION RISK]
|
|
|
|
---
|
|
|
|
## Flagged Items (Review These First)
|
|
|
|
Items rated DISPUTED or FABRICATION RISK. These need your attention before you rely on the source material.
|
|
|
|
### [C#] -- [Brief description of the claim]
|
|
|
|
- **Claim:** [The specific assertion from the target text]
|
|
- **Rating:** [DISPUTED or FABRICATION RISK]
|
|
- **Finding:** [What the verification found -- what's wrong or suspicious]
|
|
- **Source:** [URL to contradicting or relevant source]
|
|
- **Recommendation:** [What the user should do -- e.g., "Verify this citation in Westlaw" or "Remove this statistic unless you can find a primary source"]
|
|
|
|
---
|
|
|
|
## All Claims
|
|
|
|
Full results for every extracted claim, grouped by confidence rating.
|
|
|
|
### VERIFIED
|
|
|
|
#### [C#] -- [Brief description]
|
|
- **Claim:** [The assertion]
|
|
- **Source:** [URL]
|
|
- **Notes:** [Any relevant context about the source]
|
|
|
|
### PLAUSIBLE
|
|
|
|
#### [C#] -- [Brief description]
|
|
- **Claim:** [The assertion]
|
|
- **Notes:** [Why this is rated plausible rather than verified]
|
|
|
|
### UNVERIFIED
|
|
|
|
#### [C#] -- [Brief description]
|
|
- **Claim:** [The assertion]
|
|
- **Notes:** [What was searched, why nothing was found]
|
|
|
|
### DISPUTED
|
|
|
|
#### [C#] -- [Brief description]
|
|
- **Claim:** [The assertion]
|
|
- **Contradicting source:** [URL]
|
|
- **Details:** [What the source says vs. what the claim says]
|
|
|
|
### FABRICATION RISK
|
|
|
|
#### [C#] -- [Brief description]
|
|
- **Claim:** [The assertion]
|
|
- **Pattern:** [Which hallucination pattern this matches]
|
|
- **Details:** [Why this is flagged -- e.g., "citation not found in any legal database"]
|
|
|
|
---
|
|
|
|
## Internal Consistency
|
|
|
|
[Any contradictions found within the target text itself, or "No internal contradictions detected."]
|
|
|
|
---
|
|
|
|
## What Was Not Checked
|
|
|
|
[List any claims that could not be evaluated -- paywalled sources, claims requiring specialized databases, unfalsifiable assertions, etc.]
|
|
|
|
---
|
|
|
|
## Limitations
|
|
|
|
- This tool accelerates human verification; it does not replace it.
|
|
- Web search results may not include the most recent information or paywalled sources.
|
|
- The adversarial review uses the same underlying model that may have produced the original output. It catches many issues but cannot catch all of them.
|
|
- A claim rated VERIFIED means a supporting source was found, not that the claim is definitely correct. Sources can be wrong too.
|
|
- Claims rated PLAUSIBLE may still be wrong. The absence of contradicting evidence is not proof of accuracy.
|