diff --git a/CHANGELOG.md b/CHANGELOG.md index c0c486f5..d9edc4b0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,11 @@ called out in a **Breaking** section. ### Added +- `/abcd:consult` and `/abcd:ingest` skills — consult the user-level sources + corpus (confidential entries are never cited or named in public artifacts) + and ingest a URL or document into it with extracted reference metadata, + keywords, and a text-quality check. Both are thin fronts on the corpus's own + tooling and stop gracefully when no corpus exists. - A `persona_registry` record-lint rule: press-release quote attributions (`said ,`) must name a persona from the registry file the rule's `registry` key points at; unknown names are blocker findings. Configured diff --git a/skills/consult/SKILL.md b/skills/consult/SKILL.md new file mode 100644 index 00000000..2216e39f --- /dev/null +++ b/skills/consult/SKILL.md @@ -0,0 +1,80 @@ +--- +name: consult +description: Consult the local sources corpus (the user-level home's sources store, default ~/.abcd/sources) and record source→decision provenance in its append-only ledger. Use when the user says "consult sources", "check the corpus", "what do my sources say", or when a design/research decision arises where prior literature or private working material plausibly matters. Confidential sources are NEVER cited or named in public artifacts. +--- + +# Consult sources + +A local-only corpus at `~/.abcd/sources/` holds source documents (working +papers, private-repo notes, PDFs, books) the agent may **consult** but must +never **cite** publicly. Metadata lives in `sources.json` (CSL-JSON; the +`custom` block carries `confidential`, `permission_status`, `keywords`, +`aliases`). Full details: `~/.abcd/sources/README.md`. If the corpus is +absent, say so and stop — this skill never creates it. + +## Hard rule (overrides convenience, always) + +For any entry with `custom.confidential: true`: never write its title, author +names, aliases, or any identifying string into anything tracked by git or sent +anywhere external — commits, commit messages, PR/issue text, docs, code +comments, published artifacts, pasted output. This covers **identifying +paraphrase** too: do not describe a confidential source so specifically that a +reader could identify it ("a forthcoming paper showing X beats Y on Z", a +private repo's distinctive architecture) — the mechanical guards catch literal +strings only; this rule is the paraphrase layer. Refer generically ("a working +paper on X", "prior private work"). The ledger holds the real reference; +citation is the user's manual decision, made when permission exists (public +citation requires BOTH the source's permission_status AND the ledger line's +cited_publicly flag). + +Conversation with the user is fine — discuss confidential sources freely there. + +## Consult + +1. Search the corpus: `grep -ril "" ~/.abcd/sources/confidential/ + ~/.abcd/sources/public/` — try keywords, author surnames, and CSL keys. + The path IS the classification: any hit under `confidential/` falls under + the hard rule above. Each source is a folder (`//`) holding + `original.`, `text.md`, and any summaries/notes. +2. Read matched files freely for conversation; the folder's class governs + what may leave it. +3. If nothing relevant surfaces, say so — do not pad. + +To add a source, use `/abcd:ingest`. + +## Record influence (the ledger) + +Whenever a source **meaningfully influences a decision** (supports it, +contradicts it, supplies a method, or shapes background understanding — not +mere incidental reading), append ONE line to +`~/.abcd/sources/ledger/.jsonl`: + +```json +{"ts":"","repo":"","decision_ref":"","claim":"","source_key":"","locator":"","influence":"supports|contradicts|method|background","used_in":[""],"cited_publicly":false} +``` + +`used_in` makes acknowledgment machine-readable in both directions: an idea +is traced to its source even when the consuming document only paraphrases +(public sources) or must stay silent (confidential sources). Fill it whenever +the influence landed in an identifiable document, not just a conversation. + +Then commit in the corpus repo: +`git -C ~/.abcd/sources add -A && git -C ~/.abcd/sources commit -m "ledger(): → "` + +The ledger is append-only: corrections are new lines, never edits. +`cited_publicly` is always written `false`; only the user flips it, by hand. + +Always tell the user in conversation which key was recorded against which +decision, so they can decide about citing. + +## Guard wiring + +- On first use in a repo (and after any confidential source is added), run + `~/.abcd/sources/bin/sync-banlist `. It maintains a generated + block in the repo's untracked `.abcd/.work.local/private-names.txt`, which + the repo's pre-commit guard reads — leakage is then blocked mechanically, + not just by this skill's rule. +- Before any document that drew on confidential material is committed, posted, + or otherwise shared, run `~/.abcd/sources/bin/cite-guard ` (exit 1 = + confidential identifier present; its report names only the CSL key, so the + report itself is safe to relay). diff --git a/skills/ingest/SKILL.md b/skills/ingest/SKILL.md new file mode 100644 index 00000000..0cb93214 --- /dev/null +++ b/skills/ingest/SKILL.md @@ -0,0 +1,79 @@ +--- +name: ingest +description: Ingest a URL or document into the local sources corpus (the user-level home's sources store, default ~/.abcd/sources) with extracted reference metadata, keywords, and a text-quality check. Use when the user says "ingest this", "add this source/URL/paper to the corpus", "register this source", or hands over a document/link to be stored. For consulting the corpus or recording provenance, use /abcd:consult. +--- + +# Ingest a source + +Register a URL or local document in the corpus at `~/.abcd/sources/`. The +script does the deterministic half (fetch, convert, store, guard); you do the +judgment half (clean metadata, real keywords, confidentiality, quality check). +Corpus contract: `~/.abcd/sources/README.md`. The confidentiality hard rule +from `/abcd:consult` applies here in full. If the corpus is absent, say so and +stop — this skill never creates it. + +## 1. Read the document first + +- URL: WebFetch it (metadata + a skim of the content). A local fetch for + storage happens in step 3 regardless — WebFetch is for your judgment only. +- Local file: Read it (PDFs via the Read tool; big files: first pages suffice). + +Extract: exact title (no site suffixes), authors (each as `Family, Given`), +publication year, venue, canonical URL. Map the type to CSL: `article-journal` +(papers), `webpage` (posts/docs), `book`, `report` (white papers, internal +docs), `motion_picture` (video). + +## 2. Decide class and key + +- **Class.** Web content is `public` by default. Signals for + `--confidential`: the user says so; it is their own unpublished/submitted + work; internal or NDA material; AI-generated content (never citable); a + private repo's documentation. When confidential, ask the user for every + identifying name variant (`--aliases` — repo names, codenames, domains) and + pick `--permission`: `no-public-citation`, `internal-never-cite`, + `ai-generated-never-cite`, or `ask-author`. +- **Titles of confidential entries become banned phrases** (whole title, + whitespace-flexible). For internal artifacts whose natural title reads like + normal prose, register a distinctive title instead — e.g. + "meeting-notes-2026-07 (internal)" — so the ban cannot trip on legitimate + text. +- **Key**: ``, lowercase ASCII (e.g. + `naur1985theory`). Check uniqueness: `jq -r '.[].id' ~/.abcd/sources/sources.json`. + +## 3. Register + +```sh +~/.abcd/sources/bin/add-source --key --title "" \ + --type <csl-type> [--author "Family, Given"]... [--year YYYY] \ + --keywords "<k1, k2, ...>" [--aliases "a,b"] [--confidential] \ + [--permission <status>] [--url <url>] [file] +``` + +- URL only (no file): `--url` makes the script fetch and store the page. +- Local file + known URL: pass both; the URL is recorded, the file stored. +- **Keywords are the retrieval surface** — write 5–10 from having actually + read the piece: topics, named tools/techniques, the claims it makes. Never + generic filler ("AI", "software"). + +(`abcd-ingest <url-or-file>` is the human quick path to the same registrar; +prefer explicit `add-source` flags here since you have better metadata.) + +## 4. Quality-check the extraction + +Check `~/.abcd/sources/<class>/<key>/text.md` — word count sane, real prose +present. Known failure modes: `.mhtml` (unsupported → stub; extract by hand), +saved SPA/artifact pages whose content sits HTML-escaped in a wrapper +(unescape entities, `pandoc -t gfm`, rebuild text.md below its frontmatter, +commit in the corpus repo). If the source is webloc/link-only there is no body +— a metadata+URL stub is correct. + +## 5. Close out + +- Confidential ingest → run `~/.abcd/sources/bin/sync-banlist <repo-root>` in + every guarded repo the session touches. +- If the user wants a summary or review kept: write it to the source's own + folder (`summary.md`, notes as siblings) — derived artifacts inherit the + source's class by location, never anywhere else. +- If the ingest was motivated by a live decision, record the influence edge in + the ledger per `/abcd:consult`. +- Tell the user the key and class you registered.