mirror of
https://github.com/github/awesome-copilot.git
synced 2026-03-12 04:05:12 +00:00
eb7d223446
* Add doublecheck plugin: three-layer verification pipeline for AI output Adds a new plugin that helps users verify AI-generated content before acting on it. Designed for sensitive contexts (legal, medical, financial, compliance) where hallucinations carry real consequences. Three verification layers: - Self-Audit: extracts verifiable claims, checks internal consistency - Source Verification: web searches per claim, produces URLs for human review - Adversarial Review: assumes errors exist, checks hallucination patterns Supports persistent mode (auto-verifies every factual response inline) and one-shot mode (full report on specific text). Confidence ratings: VERIFIED, PLAUSIBLE, UNVERIFIED, DISPUTED, FABRICATION RISK. Includes: - Skill (skills/doublecheck/) with bundled report template - Agent (agents/doublecheck.agent.md) for interactive verification - Plugin package (plugins/doublecheck/) bundling both Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address review: fix tools YAML format, remove materialized artifacts - Fix tools frontmatter in agents/doublecheck.agent.md to use standard YAML list format instead of flow sequence with trailing comma - Remove plugins/doublecheck/agents/ and plugins/doublecheck/skills/ from tracking; these paths are in .gitignore as CI-materialized artifacts that should not be committed Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2.7 KiB
2.7 KiB
Verification Report
Summary
Text verified: [Brief description of what was checked] Claims extracted: [N total] Breakdown:
| Rating | Count |
|---|---|
| VERIFIED | |
| PLAUSIBLE | |
| UNVERIFIED | |
| DISPUTED | |
| FABRICATION RISK |
Items requiring attention: [N items rated DISPUTED or FABRICATION RISK]
Flagged Items (Review These First)
Items rated DISPUTED or FABRICATION RISK. These need your attention before you rely on the source material.
[C#] -- [Brief description of the claim]
- Claim: [The specific assertion from the target text]
- Rating: [DISPUTED or FABRICATION RISK]
- Finding: [What the verification found -- what's wrong or suspicious]
- Source: [URL to contradicting or relevant source]
- Recommendation: [What the user should do -- e.g., "Verify this citation in Westlaw" or "Remove this statistic unless you can find a primary source"]
All Claims
Full results for every extracted claim, grouped by confidence rating.
VERIFIED
[C#] -- [Brief description]
- Claim: [The assertion]
- Source: [URL]
- Notes: [Any relevant context about the source]
PLAUSIBLE
[C#] -- [Brief description]
- Claim: [The assertion]
- Notes: [Why this is rated plausible rather than verified]
UNVERIFIED
[C#] -- [Brief description]
- Claim: [The assertion]
- Notes: [What was searched, why nothing was found]
DISPUTED
[C#] -- [Brief description]
- Claim: [The assertion]
- Contradicting source: [URL]
- Details: [What the source says vs. what the claim says]
FABRICATION RISK
[C#] -- [Brief description]
- Claim: [The assertion]
- Pattern: [Which hallucination pattern this matches]
- Details: [Why this is flagged -- e.g., "citation not found in any legal database"]
Internal Consistency
[Any contradictions found within the target text itself, or "No internal contradictions detected."]
What Was Not Checked
[List any claims that could not be evaluated -- paywalled sources, claims requiring specialized databases, unfalsifiable assertions, etc.]
Limitations
- This tool accelerates human verification; it does not replace it.
- Web search results may not include the most recent information or paywalled sources.
- The adversarial review uses the same underlying model that may have produced the original output. It catches many issues but cannot catch all of them.
- A claim rated VERIFIED means a supporting source was found, not that the claim is definitely correct. Sources can be wrong too.
- Claims rated PLAUSIBLE may still be wrong. The absence of contradicting evidence is not proof of accuracy.