feat: /abcd:consult and /abcd:ingest skills over the sources corpus

CHANGELOG.md

@@ -12,6 +12,11 @@ called out in a **Breaking** section.
 
 ### Added
 
+- `/abcd:consult` and `/abcd:ingest` skills — consult the user-level sources
+  corpus (confidential entries are never cited or named in public artifacts)
+  and ingest a URL or document into it with extracted reference metadata,
+  keywords, and a text-quality check. Both are thin fronts on the corpus's own
+  tooling and stop gracefully when no corpus exists.
 - A `persona_registry` record-lint rule: press-release quote attributions
   (`said <Name>,`) must name a persona from the registry file the rule's
   `registry` key points at; unknown names are blocker findings. Configured

skills/consult/SKILL.md

@@ -0,0 +1,80 @@
+---
+name: consult
+description: Consult the local sources corpus (the user-level home's sources store, default ~/.abcd/sources) and record source→decision provenance in its append-only ledger. Use when the user says "consult sources", "check the corpus", "what do my sources say", or when a design/research decision arises where prior literature or private working material plausibly matters. Confidential sources are NEVER cited or named in public artifacts.
+---
+
+# Consult sources
+
+A local-only corpus at `~/.abcd/sources/` holds source documents (working
+papers, private-repo notes, PDFs, books) the agent may **consult** but must
+never **cite** publicly. Metadata lives in `sources.json` (CSL-JSON; the
+`custom` block carries `confidential`, `permission_status`, `keywords`,
+`aliases`). Full details: `~/.abcd/sources/README.md`. If the corpus is
+absent, say so and stop — this skill never creates it.
+
+## Hard rule (overrides convenience, always)
+
+For any entry with `custom.confidential: true`: never write its title, author
+names, aliases, or any identifying string into anything tracked by git or sent
+anywhere external — commits, commit messages, PR/issue text, docs, code
+comments, published artifacts, pasted output. This covers **identifying
+paraphrase** too: do not describe a confidential source so specifically that a
+reader could identify it ("a forthcoming paper showing X beats Y on Z", a
+private repo's distinctive architecture) — the mechanical guards catch literal
+strings only; this rule is the paraphrase layer. Refer generically ("a working
+paper on X", "prior private work"). The ledger holds the real reference;
+citation is the user's manual decision, made when permission exists (public
+citation requires BOTH the source's permission_status AND the ledger line's
+cited_publicly flag).
+
+Conversation with the user is fine — discuss confidential sources freely there.
+
+## Consult
+
+1. Search the corpus: `grep -ril "<term>" ~/.abcd/sources/confidential/
+   ~/.abcd/sources/public/` — try keywords, author surnames, and CSL keys.
+   The path IS the classification: any hit under `confidential/` falls under
+   the hard rule above. Each source is a folder (`<class>/<key>/`) holding
+   `original.<ext>`, `text.md`, and any summaries/notes.
+2. Read matched files freely for conversation; the folder's class governs
+   what may leave it.
+3. If nothing relevant surfaces, say so — do not pad.
+
+To add a source, use `/abcd:ingest`.
+
+## Record influence (the ledger)
+
+Whenever a source **meaningfully influences a decision** (supports it,
+contradicts it, supplies a method, or shapes background understanding — not
+mere incidental reading), append ONE line to
+`~/.abcd/sources/ledger/<repo>.jsonl`:
+
+```json
+{"ts":"<UTC ISO-8601>","repo":"<repo>","decision_ref":"<DECISIONS.md date | ADR id | intent id | free text>","claim":"<what was decided/claimed>","source_key":"<CSL id>","locator":"<pp./§ if known>","influence":"supports|contradicts|method|background","used_in":["<repo-relative path(s) of the consuming document(s)>"],"cited_publicly":false}
+```
+
+`used_in` makes acknowledgment machine-readable in both directions: an idea
+is traced to its source even when the consuming document only paraphrases
+(public sources) or must stay silent (confidential sources). Fill it whenever
+the influence landed in an identifiable document, not just a conversation.
+
+Then commit in the corpus repo:
+`git -C ~/.abcd/sources add -A && git -C ~/.abcd/sources commit -m "ledger(<repo>): <source_key> → <short decision>"`
+
+The ledger is append-only: corrections are new lines, never edits.
+`cited_publicly` is always written `false`; only the user flips it, by hand.
+
+Always tell the user in conversation which key was recorded against which
+decision, so they can decide about citing.
+
+## Guard wiring
+
+- On first use in a repo (and after any confidential source is added), run
+  `~/.abcd/sources/bin/sync-banlist <repo-root>`. It maintains a generated
+  block in the repo's untracked `.abcd/.work.local/private-names.txt`, which
+  the repo's pre-commit guard reads — leakage is then blocked mechanically,
+  not just by this skill's rule.
+- Before any document that drew on confidential material is committed, posted,
+  or otherwise shared, run `~/.abcd/sources/bin/cite-guard <file>` (exit 1 =
+  confidential identifier present; its report names only the CSL key, so the
+  report itself is safe to relay).

skills/ingest/SKILL.md

@@ -0,0 +1,79 @@
+---
+name: ingest
+description: Ingest a URL or document into the local sources corpus (the user-level home's sources store, default ~/.abcd/sources) with extracted reference metadata, keywords, and a text-quality check. Use when the user says "ingest this", "add this source/URL/paper to the corpus", "register this source", or hands over a document/link to be stored. For consulting the corpus or recording provenance, use /abcd:consult.
+---
+
+# Ingest a source
+
+Register a URL or local document in the corpus at `~/.abcd/sources/`. The
+script does the deterministic half (fetch, convert, store, guard); you do the
+judgment half (clean metadata, real keywords, confidentiality, quality check).
+Corpus contract: `~/.abcd/sources/README.md`. The confidentiality hard rule
+from `/abcd:consult` applies here in full. If the corpus is absent, say so and
+stop — this skill never creates it.
+
+## 1. Read the document first
+
+- URL: WebFetch it (metadata + a skim of the content). A local fetch for
+  storage happens in step 3 regardless — WebFetch is for your judgment only.
+- Local file: Read it (PDFs via the Read tool; big files: first pages suffice).
+
+Extract: exact title (no site suffixes), authors (each as `Family, Given`),
+publication year, venue, canonical URL. Map the type to CSL: `article-journal`
+(papers), `webpage` (posts/docs), `book`, `report` (white papers, internal
+docs), `motion_picture` (video).
+
+## 2. Decide class and key
+
+- **Class.** Web content is `public` by default. Signals for
+  `--confidential`: the user says so; it is their own unpublished/submitted
+  work; internal or NDA material; AI-generated content (never citable); a
+  private repo's documentation. When confidential, ask the user for every
+  identifying name variant (`--aliases` — repo names, codenames, domains) and
+  pick `--permission`: `no-public-citation`, `internal-never-cite`,
+  `ai-generated-never-cite`, or `ask-author`.
+- **Titles of confidential entries become banned phrases** (whole title,
+  whitespace-flexible). For internal artifacts whose natural title reads like
+  normal prose, register a distinctive title instead — e.g.
+  "meeting-notes-2026-07 (internal)" — so the ban cannot trip on legitimate
+  text.
+- **Key**: `<authorfamily><year><distinctiveword>`, lowercase ASCII (e.g.
+  `naur1985theory`). Check uniqueness: `jq -r '.[].id' ~/.abcd/sources/sources.json`.
+
+## 3. Register
+
+```sh
+~/.abcd/sources/bin/add-source --key <key> --title "<title>" \
+  --type <csl-type> [--author "Family, Given"]... [--year YYYY] \
+  --keywords "<k1, k2, ...>" [--aliases "a,b"] [--confidential] \
+  [--permission <status>] [--url <url>] [file]
+```
+
+- URL only (no file): `--url` makes the script fetch and store the page.
+- Local file + known URL: pass both; the URL is recorded, the file stored.
+- **Keywords are the retrieval surface** — write 5–10 from having actually
+  read the piece: topics, named tools/techniques, the claims it makes. Never
+  generic filler ("AI", "software").
+
+(`abcd-ingest <url-or-file>` is the human quick path to the same registrar;
+prefer explicit `add-source` flags here since you have better metadata.)
+
+## 4. Quality-check the extraction
+
+Check `~/.abcd/sources/<class>/<key>/text.md` — word count sane, real prose
+present. Known failure modes: `.mhtml` (unsupported → stub; extract by hand),
+saved SPA/artifact pages whose content sits HTML-escaped in a wrapper
+(unescape entities, `pandoc -t gfm`, rebuild text.md below its frontmatter,
+commit in the corpus repo). If the source is webloc/link-only there is no body
+— a metadata+URL stub is correct.
+
+## 5. Close out
+
+- Confidential ingest → run `~/.abcd/sources/bin/sync-banlist <repo-root>` in
+  every guarded repo the session touches.
+- If the user wants a summary or review kept: write it to the source's own
+  folder (`summary.md`, notes as siblings) — derived artifacts inherit the
+  source's class by location, never anywhere else.
+- If the ingest was motivated by a live decision, record the influence edge in
+  the ledger per `/abcd:consult`.
+- Tell the user the key and class you registered.