Add source-grounded evidence map skill 🤖🤖🤖 (#2505)

* Add source-grounded evidence map skill

* Bundle offline evidence map validation

* Avoid implicit package execution
This commit is contained in:
alsoleg89
2026-08-05 08:17:55 +03:00
committed by GitHub
parent f83a8a942c
commit 37ade4b521
6 changed files with 857 additions and 0 deletions
+1
View File
@@ -92,6 +92,7 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md#adding-skills) for guidelines on how to
| [breakdown-plan](../skills/breakdown-plan/SKILL.md)<br />`gh skills install github/awesome-copilot breakdown-plan` | Issue Planning and Automation prompt that generates comprehensive project plans with Epic > Feature > Story/Enabler > Test hierarchy, dependencies, priorities, and automated tracking. | None |
| [breakdown-test](../skills/breakdown-test/SKILL.md)<br />`gh skills install github/awesome-copilot breakdown-test` | Test Planning and Quality Assurance prompt that generates comprehensive test strategies, task breakdowns, and quality validation plans for GitHub projects. | None |
| [bug-reproduction-brief](../skills/bug-reproduction-brief/SKILL.md)<br />`gh skills install github/awesome-copilot bug-reproduction-brief` | Turn a vague, intermittent, or environment-specific bug report into a minimal evidence-backed reproduction before proposing a fix. | None |
| [build-evidence-map](../skills/build-evidence-map/SKILL.md)<br />`gh skills install github/awesome-copilot build-evidence-map` | Build an auditable evidence map for a contested technical choice, research synthesis, proposal review, or consequential decision. Use when Copilot must preserve supporting, contradicting, qualifying, and missing evidence with exact source regions instead of collapsing disagreement into prose. | `references/evidence-ladder.md`<br />`references/map-schema.md`<br />`scripts/contract.mjs`<br />`scripts/validate.mjs` |
| [centos-linux-triage](../skills/centos-linux-triage/SKILL.md)<br />`gh skills install github/awesome-copilot centos-linux-triage` | Triage and resolve CentOS issues using RHEL-compatible tooling, SELinux-aware practices, and firewalld. | None |
| [chrome-devtools](../skills/chrome-devtools/SKILL.md)<br />`gh skills install github/awesome-copilot chrome-devtools` | Expert-level browser automation, debugging, and performance analysis using Chrome DevTools MCP. Use for interacting with web pages, capturing screenshots, analyzing network traffic, and profiling performance. | None |
| [cli-mastery](../skills/cli-mastery/SKILL.md)<br />`gh skills install github/awesome-copilot cli-mastery` | Interactive training for the GitHub Copilot CLI. Guided lessons, quizzes, scenario challenges, and a full reference covering slash commands, shortcuts, modes, agents, skills, MCP, and configuration. Say "cliexpert" to start. | `references/final-exam.md`<br />`references/module-1-slash-commands.md`<br />`references/module-2-keyboard-shortcuts.md`<br />`references/module-3-modes.md`<br />`references/module-4-agents.md`<br />`references/module-5-skills.md`<br />`references/module-6-mcp.md`<br />`references/module-7-advanced.md`<br />`references/module-8-configuration.md`<br />`references/scenarios.md` |
+107
View File
@@ -0,0 +1,107 @@
---
name: build-evidence-map
description: 'Build an auditable evidence map for a contested technical choice, research synthesis, proposal review, or consequential decision. Use when Copilot must preserve supporting, contradicting, qualifying, and missing evidence with exact source regions instead of collapsing disagreement into prose.'
---
# Build Evidence Map
Turn one contested question into a portable decision artifact that shows what
supports the current position, what pushes against it, and what remains unknown.
Do not use a graph to decorate an answer that has not been sourced.
For a simple factual claim or a general fact-checking request, use a verification
workflow such as `doublecheck` instead. Use this skill when the relationships
between evidence, intermediate claims, trade-offs, and missing facts matter.
## Workflow
1. **Frame one decision.** Write one falsifiable question and one provisional
position. Narrow the question until a reader can identify what action or
belief the map is testing.
2. **Collect bounded source regions.** Prefer direct observations and primary
sources. Record the URL or absolute local path, publisher, publication date,
retrieval date, section/page/line/timestamp locator, and a short checkable
excerpt. Read [references/evidence-ladder.md](references/evidence-ladder.md)
when source quality is disputed.
3. **Atomize the reasoning.** Create only four node types:
- `position`: the single current verdict;
- `claim`: an intermediate proposition;
- `evidence`: a faithful statement of one source region;
- `unknown`: a specific missing fact that could change the verdict.
4. **Type every edge.** Use `supports`, `contradicts`, `qualifies`, or
`missing`. Add a plain-language note explaining why the source node bears on
the target. Topical similarity is not support. Different scope, date, or
population is not automatically a contradiction.
5. **Preserve counterevidence.** Do not delete contrary evidence because the
provisional verdict survives it. Represent scope differences with
`qualifies` edges.
6. **Express uncertainty structurally.** Do not invent confidence percentages.
Add an `unknown`, narrow the position, or qualify a claim.
7. **Write UTF-8 JSON** with a `.doubt.json` suffix. Follow
[references/map-schema.md](references/map-schema.md). Keep IDs short,
stable, and semantic.
8. **Validate fail-closed.** Resolve
`scripts/validate.mjs` relative to this `SKILL.md`, then run it with Node.js
18 or newer:
```bash
node <skill-directory>/scripts/validate.mjs decision.doubt.json
```
The bundled validator uses only Node.js built-ins and does not require npm or
network access. Fix every finding before reporting success. Only say the map
is valid when the command exits `0` and prints `VALID` followed by a
64-character receipt. A file hash, node count, JSON parse, or manual schema
review is not a Doubt receipt. If deterministic validation cannot run, report
that block instead of inventing success.
Render the validated map only when the user has already installed
`doubt-ai@0.8.0`; do not install or execute a remote package implicitly:
```bash
doubt map decision.doubt.json --out decision.html
```
9. **Verify source snapshots only with explicit network permission.** The
following command retrieves each recorded HTTP(S) source and fails closed if
an excerpt cannot be matched:
```bash
doubt verify decision.doubt.json \
--out decision.verified.doubt.json
```
Never run this command implicitly. Local file verification does not use the
network. Do not write a `verification` object by hand or hide a mismatch.
10. **Inspect the deliverable.** Confirm that the question, verdict,
counterevidence, unknowns, edge notes, and exact source regions remain
readable. Treat JSON as the canonical editable artifact; HTML is a
shareable view.
## Quality gates
A finished map must satisfy all of these:
- exactly one `position` has incoming reasoning;
- every evidence node names one source and participates in an edge;
- every source is used and has dates, a bounded locator, and a substantive
excerpt;
- every non-position node has a directed path to the position;
- the reasoning graph has no duplicate edges or directed cycles;
- contrary or qualifying evidence is present when the source set contains it;
- each decision-changing gap is an explicit `unknown` node;
- every edge note explains support, contradiction, qualification, or absence;
- the verdict is no broader than the evidence.
## Deliver the result
Report:
- the current position in one sentence;
- the strongest counterevidence or qualification;
- the most important unresolved unknown;
- paths to the canonical JSON and any rendered HTML;
- whether deterministic validation and explicit source verification ran.
Never describe a structurally valid map as proven true. Validation establishes
traceability and graph integrity; source quality and inference quality still
require human review.
@@ -0,0 +1,40 @@
# Evidence ladder
Use the strongest evidence practical for the decision. A higher class can still
be stale, irrelevant, or too broad for the nearby claim.
1. **Direct current observation** — reproduced behavior, command output,
inspected artifact, or measured result.
2. **Authoritative primary source** — official specification, dataset, law,
documentation, first-party repository, or original research.
3. **Independent corroboration** — competent sources with distinct underlying
evidence.
4. **Explicit inference** — a conclusion whose premises and assumptions are
visible in the map.
5. **Weak proxy** — related metric, benchmark, anecdote, or test that does not
exercise the exact claim.
6. **Unsupported assertion** — confidence, repetition, or polished language
without evidence.
## Source-region test
Before creating an evidence node, answer:
- What exact sentence, table, command output, page, section, or line range is
being relied on?
- Does it entail the node text, or merely discuss the same subject?
- Is its date and version appropriate for the claim?
- Is the evidence independent, or copied from another cited source?
- What context would reverse or narrow the interpretation?
If the exact region cannot be located, create an `unknown` node instead of an
evidence node.
## Edge test
| Relation | Use when | Common counterfeit |
| --- | --- | --- |
| `supports` | The source increases reason to accept the target | Topical similarity |
| `contradicts` | Both cannot hold under the same scope and conditions | Different dates or populations |
| `qualifies` | The source narrows scope, strength, or applicability | Hiding inconvenient evidence |
| `missing` | A specific absent fact blocks or could reverse the target | Generic “more research needed” |
@@ -0,0 +1,101 @@
# Evidence map schema
The canonical artifact is UTF-8 JSON. Use a `.doubt.json` suffix when practical.
```json
{
"title": "Short artifact title",
"question": "One decision-changing question?",
"updatedAt": "YYYY-MM-DD",
"verdict": "A provisional, evidence-bounded answer.",
"nodes": [
{
"id": "current-position",
"type": "position",
"label": "Current position",
"text": "The proposition represented by this node."
},
{
"id": "primary-observation",
"type": "evidence",
"label": "Observed result",
"text": "A faithful statement of the source region.",
"sourceId": "source-1"
},
{
"id": "missing-baseline",
"type": "unknown",
"label": "Missing baseline",
"text": "The exact absent fact and why it matters."
}
],
"edges": [
{
"from": "primary-observation",
"to": "current-position",
"relation": "supports",
"note": "Why the observation increases reason to accept the position."
},
{
"from": "missing-baseline",
"to": "current-position",
"relation": "missing",
"note": "Why this missing baseline could reverse the position."
}
],
"sources": [
{
"id": "source-1",
"title": "Source title",
"url": "https://example.com/source",
"publisher": "Publisher",
"date": "YYYY-MM-DD",
"retrievedAt": "YYYY-MM-DD",
"locator": "Section: Results, p. 7, § 2.1, L12-L18, or 00:04:31",
"excerpt": "A short, checkable excerpt or bounded source-region description."
}
]
}
```
## Invariants
- Allowed node types: `position`, `claim`, `evidence`, `unknown`.
- Allowed relations: `supports`, `contradicts`, `qualifies`, `missing`.
- Exactly one `position` node is required.
- Evidence nodes require `sourceId`.
- Every evidence node must be the `from` side of at least one edge.
- Every non-position node must have a directed path to the position.
- Duplicate reasoning edges and directed cycles are rejected.
- Every source must be used by an evidence node.
- Every edge needs a plain-language `note`.
- Map and source dates are real ISO calendar dates; source dates cannot be later
than `updatedAt`.
- Every source records `retrievedAt`. Receipts cover that value and the recorded
excerpt, not the mutable bytes currently served by the URL.
- Locators identify a bounded section, page, line range, or timestamp.
- Excerpts contain 40500 characters of varied, checkable content; repeated
filler is invalid.
- `confidence` fields are invalid. Use an `unknown` node or a qualified claim.
## Optional verification record
Only a successful explicit source-verification command may add this object to a
source:
```json
{
"verification": {
"status": "verified",
"method": "normalized-excerpt-match",
"checkedAt": "YYYY-MM-DDTHH:mm:ss.sssZ",
"contentSha256": "64 lowercase hexadecimal characters",
"excerptSha256": "64 lowercase hexadecimal characters",
"finalUrl": "The checked URL or absolute local path",
"locatorStatus": "matched"
}
}
```
`locatorStatus` may be `not-machine-checked` for page, section, and timestamp
locators. Do not treat it as proof that the region was manually confirmed.
@@ -0,0 +1,517 @@
export const NODE_TYPES = new Set(["position", "claim", "evidence", "unknown"]);
export const RELATIONS = new Set(["supports", "contradicts", "qualifies", "missing"]);
const ISO_DATE = /^(\d{4})-(\d{2})-(\d{2})$/;
const ISO_UTC_TIMESTAMP = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.\d{1,3})?Z$/;
const LOCATOR_PATTERNS = [
/\bp(?:age)?\.?\s*\d+(?:\s*[-]\s*\d+)?\b/i,
/§\s*[\p{L}\p{N}][\p{L}\p{N}._-]*/u,
/\bL\d+(?:\s*[-]\s*L?\d+)?\b/i,
/\blines?\s+\d+(?:\s*[-]\s*\d+)?\b/i,
/\b(?:\d{1,2}:)?\d{2}:\d{2}(?:\s*[-]\s*(?:\d{1,2}:)?\d{2}:\d{2})?\b/,
/^(?:section|chapter|heading)\s*(?::|§)\s*\S.{1,}$/i,
];
export class MapValidationError extends Error {
constructor(findings) {
super(`Evidence map is invalid (${findings.length} ${findings.length === 1 ? "finding" : "findings"}).`);
this.name = "MapValidationError";
this.findings = findings;
}
}
function canonical(value) {
if (Array.isArray(value)) return value.map(canonical);
if (!value || typeof value !== "object") return value;
return Object.fromEntries(
Object.keys(value)
.sort()
.map((key) => [key, canonical(value[key])]),
);
}
export function canonicalJson(map) {
return JSON.stringify(canonical(map));
}
export function receiptPayload(map, sourceSnapshots) {
return {
contract: "doubt-evidence-receipt-v1",
map,
sourceSnapshots,
};
}
function finding(path, rule, message) {
return { path, rule, message };
}
function parseIsoDate(value) {
if (typeof value !== "string") return null;
const match = value.match(ISO_DATE);
if (!match) return null;
const [, year, month, day] = match.map(Number);
const time = Date.UTC(year, month - 1, day);
const date = new Date(time);
if (
date.getUTCFullYear() !== year
|| date.getUTCMonth() !== month - 1
|| date.getUTCDate() !== day
) return null;
return time;
}
function retrievalDate(value) {
const date = parseIsoDate(value);
if (date !== null) return date;
if (typeof value !== "string") return null;
const match = value.match(ISO_UTC_TIMESTAMP);
if (!match) return null;
const [, year, month, day, hour, minute, second] = match.map(Number);
if (hour > 23 || minute > 59 || second > 59) return null;
const dayValue = parseIsoDate(
`${String(year).padStart(4, "0")}-${String(month).padStart(2, "0")}-${String(day).padStart(2, "0")}`,
);
return dayValue === null ? null : dayValue;
}
function validUtcTimestamp(value) {
if (typeof value !== "string") return false;
const match = value.match(ISO_UTC_TIMESTAMP);
if (!match) return false;
const [, year, month, day, hour, minute, second] = match.map(Number);
if (hour > 23 || minute > 59 || second > 59) return false;
return parseIsoDate(
`${String(year).padStart(4, "0")}-${String(month).padStart(2, "0")}-${String(day).padStart(2, "0")}`,
) !== null;
}
function boundedLocator(value) {
return typeof value === "string" && LOCATOR_PATTERNS.some((pattern) => pattern.test(value.trim()));
}
function sourceLocation(value) {
return typeof value === "string" && /^(?:https?:\/\/|file:\/\/|\.\.?[\\/]|[\\/]|[A-Za-z]:[\\/])/.test(value);
}
function substantiveExcerpt(value) {
if (typeof value !== "string") return false;
const symbols = value.toLowerCase().match(/[\p{L}\p{N}]/gu) || [];
return new Set(symbols).size >= 6;
}
function reaches(start, target, adjacency, seen = new Set()) {
if (start === target) return true;
if (seen.has(start)) return false;
seen.add(start);
return (adjacency.get(start) || []).some((next) => reaches(next, target, adjacency, seen));
}
export function inspectMapContract(map) {
const findings = [];
if (!map || typeof map !== "object" || Array.isArray(map)) {
return {
findings: [finding("$", "map-type", "The map must be a JSON object.")],
metrics: { claims: 0, contradictions: 0, evidence: 0, sources: 0, unknowns: 0 },
receipt: null,
valid: false,
};
}
for (const key of ["title", "question", "verdict", "updatedAt"]) {
if (!map[key] || typeof map[key] !== "string") {
findings.push(finding(`$.${key}`, "required-field", `${key} must be a non-empty string.`));
}
}
const updatedAt = parseIsoDate(map.updatedAt);
if (typeof map.updatedAt === "string" && updatedAt === null) {
findings.push(
finding("$.updatedAt", "map-date", "updatedAt must be a real calendar date in YYYY-MM-DD format."),
);
}
if (!Array.isArray(map.nodes) || map.nodes.length === 0) {
findings.push(finding("$.nodes", "required-nodes", "nodes must be a non-empty array."));
}
if (!Array.isArray(map.edges)) {
findings.push(finding("$.edges", "required-edges", "edges must be an array."));
}
if (!Array.isArray(map.sources)) {
findings.push(finding("$.sources", "required-sources", "sources must be an array."));
}
const nodes = Array.isArray(map.nodes) ? map.nodes : [];
const edges = Array.isArray(map.edges) ? map.edges : [];
const sources = Array.isArray(map.sources) ? map.sources : [];
const nodeIds = new Set();
const sourceIds = new Set();
for (const [index, node] of nodes.entries()) {
const base = `$.nodes[${index}]`;
if (!node || typeof node !== "object" || Array.isArray(node)) {
findings.push(finding(base, "node-type", "Each node must be an object."));
continue;
}
if (!node.id || typeof node.id !== "string") {
findings.push(finding(`${base}.id`, "node-id", "Each node needs a string id."));
} else if (nodeIds.has(node.id)) {
findings.push(finding(`${base}.id`, "duplicate-node", `Duplicate node id: ${node.id}.`));
} else {
nodeIds.add(node.id);
}
if (!NODE_TYPES.has(node.type)) {
findings.push(
finding(
`${base}.type`,
"node-type",
`Node type must be one of: ${[...NODE_TYPES].join(", ")}.`,
),
);
}
for (const key of ["label", "text"]) {
if (!node[key] || typeof node[key] !== "string") {
findings.push(finding(`${base}.${key}`, "node-copy", `${key} must be a non-empty string.`));
}
}
if (node.confidence != null) {
findings.push(
finding(
`${base}.confidence`,
"false-precision",
"Confidence percentages are not supported; use an unknown or a qualified claim instead.",
),
);
}
}
for (const [index, source] of sources.entries()) {
const base = `$.sources[${index}]`;
if (!source || typeof source !== "object" || Array.isArray(source)) {
findings.push(finding(base, "source-type", "Each source must be an object."));
continue;
}
if (!source.id || typeof source.id !== "string") {
findings.push(finding(`${base}.id`, "source-id", "Each source needs a string id."));
} else if (sourceIds.has(source.id)) {
findings.push(finding(`${base}.id`, "duplicate-source", `Duplicate source id: ${source.id}.`));
} else {
sourceIds.add(source.id);
}
for (const key of ["title", "publisher", "date", "retrievedAt", "url", "locator", "excerpt"]) {
if (!source[key] || typeof source[key] !== "string") {
findings.push(
finding(`${base}.${key}`, "source-field", `${key} must be a non-empty string.`),
);
}
}
const sourceDate = parseIsoDate(source.date);
if (typeof source.date === "string" && sourceDate === null) {
findings.push(
finding(`${base}.date`, "source-date", "Source date must be a real calendar date in YYYY-MM-DD format."),
);
} else if (sourceDate !== null && updatedAt !== null && sourceDate > updatedAt) {
findings.push(
finding(`${base}.date`, "future-source-date", "Source date cannot be later than map.updatedAt."),
);
}
const retrievedAt = retrievalDate(source.retrievedAt);
if (typeof source.retrievedAt === "string" && retrievedAt === null) {
findings.push(
finding(
`${base}.retrievedAt`,
"retrieval-date",
"retrievedAt must be YYYY-MM-DD or an ISO UTC timestamp ending in Z.",
),
);
} else if (retrievedAt !== null && updatedAt !== null && retrievedAt > updatedAt) {
findings.push(
finding(`${base}.retrievedAt`, "future-retrieval", "retrievedAt cannot be later than map.updatedAt."),
);
} else if (retrievedAt !== null && sourceDate !== null && retrievedAt < sourceDate) {
findings.push(
finding(`${base}.retrievedAt`, "retrieval-before-source", "retrievedAt cannot predate the source date."),
);
}
if (typeof source.url === "string" && !sourceLocation(source.url)) {
findings.push(
finding(
`${base}.url`,
"source-url",
"Source location must be http(s), file://, or a relative or absolute local path.",
),
);
}
if (typeof source.locator === "string" && !boundedLocator(source.locator)) {
findings.push(
finding(
`${base}.locator`,
"source-locator",
"Locator must identify a bounded page, section, line range, or timestamp (for example p. 7, § 2.1, L12-L18, Section: Results, or 00:04:31).",
),
);
}
if (typeof source.excerpt === "string" && source.excerpt.trim().length < 40) {
findings.push(
finding(
`${base}.excerpt`,
"thin-excerpt",
"Source excerpt must contain at least 40 characters of checkable context.",
),
);
}
if (typeof source.excerpt === "string" && source.excerpt.trim().length > 500) {
findings.push(
finding(
`${base}.excerpt`,
"oversized-excerpt",
"Keep source excerpts under 500 characters and link to the full source.",
),
);
}
if (
typeof source.excerpt === "string"
&& source.excerpt.trim().length >= 40
&& source.excerpt.trim().length <= 500
&& !substantiveExcerpt(source.excerpt)
) {
findings.push(
finding(
`${base}.excerpt`,
"low-information-excerpt",
"Source excerpt must contain varied, checkable content rather than repeated filler.",
),
);
}
if (source.verification != null) {
const verification = source.verification;
const verificationBase = `${base}.verification`;
if (!verification || typeof verification !== "object" || Array.isArray(verification)) {
findings.push(
finding(verificationBase, "verification-type", "verification must be an object."),
);
} else {
if (verification.status !== "verified") {
findings.push(
finding(`${verificationBase}.status`, "verification-status", "Verification status must be verified."),
);
}
if (verification.method !== "normalized-excerpt-match") {
findings.push(
finding(
`${verificationBase}.method`,
"verification-method",
"Verification method must be normalized-excerpt-match.",
),
);
}
if (!validUtcTimestamp(verification.checkedAt)) {
findings.push(
finding(
`${verificationBase}.checkedAt`,
"verification-time",
"Verification checkedAt must be an ISO UTC timestamp ending in Z.",
),
);
} else if (source.retrievedAt !== verification.checkedAt.slice(0, 10)) {
findings.push(
finding(
`${verificationBase}.checkedAt`,
"verification-retrieval-mismatch",
"A verified source retrievedAt must equal the UTC date in verification.checkedAt.",
),
);
}
for (const key of ["contentSha256", "excerptSha256"]) {
if (typeof verification[key] !== "string" || !/^[a-f0-9]{64}$/.test(verification[key])) {
findings.push(
finding(
`${verificationBase}.${key}`,
"verification-digest",
`${key} must be a lowercase SHA-256 digest.`,
),
);
}
}
if (!["matched", "not-machine-checked"].includes(verification.locatorStatus)) {
findings.push(
finding(
`${verificationBase}.locatorStatus`,
"verification-locator",
"locatorStatus must be matched or not-machine-checked.",
),
);
}
if (typeof verification.finalUrl !== "string" || !sourceLocation(verification.finalUrl)) {
findings.push(
finding(
`${verificationBase}.finalUrl`,
"verification-url",
"finalUrl must be an http(s), file://, or local path source location.",
),
);
}
}
}
}
const incoming = new Map(nodes.filter((node) => node?.id).map((node) => [node.id, 0]));
const adjacency = new Map(nodes.filter((node) => node?.id).map((node) => [node.id, []]));
const uniqueEdges = new Set();
for (const [index, edge] of edges.entries()) {
const base = `$.edges[${index}]`;
if (!edge || typeof edge !== "object" || Array.isArray(edge)) {
findings.push(finding(base, "edge-type", "Each edge must be an object."));
continue;
}
if (!nodeIds.has(edge.from)) {
findings.push(finding(`${base}.from`, "unknown-node", `Unknown from node: ${edge.from}.`));
}
if (!nodeIds.has(edge.to)) {
findings.push(finding(`${base}.to`, "unknown-node", `Unknown to node: ${edge.to}.`));
}
if (edge.from && edge.from === edge.to) {
findings.push(finding(base, "self-edge", `Node ${edge.from} cannot point to itself.`));
}
const edgeKey = `${edge.from}\0${edge.to}\0${edge.relation}`;
if (uniqueEdges.has(edgeKey)) {
findings.push(
finding(base, "duplicate-edge", "Duplicate from/to/relation edges are not allowed."),
);
} else {
uniqueEdges.add(edgeKey);
}
if (!RELATIONS.has(edge.relation)) {
findings.push(
finding(
`${base}.relation`,
"edge-relation",
`Relation must be one of: ${[...RELATIONS].join(", ")}.`,
),
);
}
if (!edge.note || typeof edge.note !== "string") {
findings.push(
finding(`${base}.note`, "edge-note", "Each reasoning edge needs a plain-language note."),
);
}
if (nodeIds.has(edge.to)) incoming.set(edge.to, (incoming.get(edge.to) || 0) + 1);
if (nodeIds.has(edge.from) && nodeIds.has(edge.to) && edge.from !== edge.to) {
adjacency.get(edge.from).push(edge.to);
}
}
for (const [index, node] of nodes.entries()) {
if (!node || typeof node !== "object") continue;
const base = `$.nodes[${index}]`;
if (node.type === "evidence" && !node.sourceId) {
findings.push(
finding(`${base}.sourceId`, "unsourced-evidence", "Evidence nodes require sourceId."),
);
}
if (node.sourceId && !sourceIds.has(node.sourceId)) {
findings.push(
finding(
`${base}.sourceId`,
"unknown-source",
`Node references unknown source: ${node.sourceId}.`,
),
);
}
if (
node.type === "evidence" &&
node.id &&
!edges.some((edge) => edge?.from === node.id)
) {
findings.push(
finding(base, "unused-evidence", "Evidence must participate in at least one reasoning edge."),
);
}
}
for (const [index, source] of sources.entries()) {
if (
source?.id &&
!nodes.some((node) => node?.type === "evidence" && node.sourceId === source.id)
) {
findings.push(
finding(
`$.sources[${index}]`,
"unused-source",
"Every source must be attached to at least one evidence node.",
),
);
}
}
const positions = nodes.filter((node) => node?.type === "position");
if (positions.length !== 1) {
findings.push(
finding("$.nodes", "position-count", "The map must contain exactly one position node."),
);
} else if (!incoming.get(positions[0].id)) {
findings.push(
finding(
`$.nodes[${nodes.indexOf(positions[0])}]`,
"unsupported-position",
"The position needs at least one incoming reasoning edge.",
),
);
} else {
for (const [index, node] of nodes.entries()) {
if (!node?.id || node.id === positions[0].id) continue;
if (!reaches(node.id, positions[0].id, adjacency)) {
findings.push(
finding(
`$.nodes[${index}]`,
"disconnected-node",
`Node ${node.id} must have a directed reasoning path to the position.`,
),
);
}
}
}
const visitState = new Map();
const cyclicNodes = new Set();
function visit(nodeId, stack = []) {
const state = visitState.get(nodeId) || 0;
if (state === 1) {
for (const member of stack.slice(stack.indexOf(nodeId))) cyclicNodes.add(member);
return;
}
if (state === 2) return;
visitState.set(nodeId, 1);
for (const next of adjacency.get(nodeId) || []) visit(next, [...stack, nodeId]);
visitState.set(nodeId, 2);
}
for (const nodeId of nodeIds) visit(nodeId);
for (const nodeId of cyclicNodes) {
const index = nodes.findIndex((node) => node?.id === nodeId);
findings.push(
finding(`$.nodes[${index}]`, "reasoning-cycle", `Node ${nodeId} participates in a reasoning cycle.`),
);
}
const metrics = {
claims: nodes.filter((node) => node?.type === "claim").length,
contradictions: new Set(
edges
.filter((edge) => edge?.relation === "contradicts")
.map((edge) => `${edge.from}\0${edge.to}\0${edge.relation}`),
).size,
evidence: nodes.filter((node) => node?.type === "evidence").length,
sources: sources.length,
unknowns: nodes.filter((node) => node?.type === "unknown").length,
};
return {
findings,
metrics,
receipt: null,
valid: findings.length === 0,
};
}
export function validateMapContract(map) {
const result = inspectMapContract(map);
if (!result.valid) throw new MapValidationError(result.findings);
return result;
}
@@ -0,0 +1,91 @@
#!/usr/bin/env node
import { createHash } from "node:crypto";
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
import { fileURLToPath } from "node:url";
import {
canonicalJson,
inspectMapContract,
receiptPayload,
} from "./contract.mjs";
function receiptFor(map) {
const sourceSnapshots = map.sources.map((source) => ({
id: source.id,
retrievedAt: source.retrievedAt,
excerptSha256: createHash("sha256").update(source.excerpt).digest("hex"),
}));
return createHash("sha256")
.update(canonicalJson(receiptPayload(map, sourceSnapshots)))
.digest("hex");
}
export function inspectOfflineMap(map) {
const result = inspectMapContract(map);
return {
...result,
receipt: result.valid ? receiptFor(map) : null,
};
}
export async function validateMapFile(file) {
let map;
try {
const input = file instanceof URL ? file : resolve(file);
map = JSON.parse(await readFile(input, "utf8"));
} catch (error) {
return {
findings: [{
path: "$",
rule: "invalid-json",
message: `Could not parse JSON: ${error.message}`,
}],
metrics: { claims: 0, contradictions: 0, evidence: 0, sources: 0, unknowns: 0 },
receipt: null,
valid: false,
};
}
return inspectOfflineMap(map);
}
function plural(count, singular, pluralForm = `${singular}s`) {
return count === 1 ? singular : pluralForm;
}
function printHuman(result) {
if (!result.valid) {
console.error(`INVALID ${result.findings.length} ${plural(result.findings.length, "finding")}`);
for (const finding of result.findings) {
console.error(` - ${finding.path} [${finding.rule}] ${finding.message}`);
}
return;
}
console.log(`VALID ${result.receipt}`);
console.log(
`${result.metrics.claims} ${plural(result.metrics.claims, "claim")} · ${result.metrics.evidence} evidence · ${result.metrics.sources} ${plural(result.metrics.sources, "source")}`,
);
console.log(
`${result.metrics.contradictions} ${plural(result.metrics.contradictions, "contradiction")} · ${result.metrics.unknowns} explicit ${plural(result.metrics.unknowns, "unknown")}`,
);
}
async function main(argv) {
if (argv.includes("--help") || argv.includes("-h")) {
console.log("Usage: node <skill-dir>/scripts/validate.mjs <map.doubt.json> [--json]");
return;
}
const file = argv.find((value) => !value.startsWith("-"));
if (!file) throw new Error("Pass a .doubt.json evidence map.");
const result = await validateMapFile(file);
if (argv.includes("--json")) console.log(JSON.stringify(result, null, 2));
else printHuman(result);
if (!result.valid) process.exitCode = 1;
}
if (process.argv[1] === fileURLToPath(import.meta.url)) {
main(process.argv.slice(2)).catch((error) => {
console.error(error.message);
process.exitCode = 1;
});
}