From 22a04ef3194cf090890cdb95a1356ea153e2f055 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Wed, 8 Jul 2026 09:04:02 +0100 Subject: [PATCH 01/11] docs: record confidential-sources provenance design (itd-76) A local-only system for consulting confidential sources without ever citing them publicly: global CSL-JSON corpus, append-only JSONL influence ledger, banlist patterns generated from confidential entries into the itd-74 private pre-commit guard, Quarto profiles for eventual paper reconstruction. Convention-first scaffold is live at user level; promotion to 'abcd source' verbs is captured as draft intent itd-76. Also records the personas principle (Alice, Bob, Carol; user is they/them) set this session. Assisted-by: Claude:claude-fable-5 --- .../drafts/itd-76-source-provenance-ledger.md | 40 +++++ ...026-07-08-confidential-sources-scaffold.md | 79 ++++++++++ .../principles/personas-alice-bob-carol.md | 20 +++ .abcd/development/research/_references.md | 24 +++ ...08-confidential-sources-provenance-sota.md | 141 ++++++++++++++++++ .abcd/work/DECISIONS.md | 7 + 6 files changed, 311 insertions(+) create mode 100644 .abcd/development/intents/drafts/itd-76-source-provenance-ledger.md create mode 100644 .abcd/development/plans/2026-07-08-confidential-sources-scaffold.md create mode 100644 .abcd/development/principles/personas-alice-bob-carol.md create mode 100644 .abcd/development/research/notes/2026-07-08-confidential-sources-provenance-sota.md diff --git a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md new file mode 100644 index 00000000..1c073031 --- /dev/null +++ b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md @@ -0,0 +1,40 @@ +--- +id: itd-76 +slug: source-provenance-ledger +spec_id: null +kind: standalone +suggested_kind: null +reclassification_history: [] +--- + +# abcd Lets You Consult Sources You Cannot Cite — and Remembers Every Debt + +## Press Release + +> **Consult any source freely; cite only by deliberate human choice — abcd keeps the ledger in between.** A developer-researcher often works from material they are not free to name in public: working papers under review, a collaborator's private repository, notes shared under an NDA. The agent should be able to *read* that material when it bears on a decision — and must never *cite* it in anything published. abcd manages this as three coupled pieces. A **local-only corpus** (a user-level directory outside every repo, itself a no-remote git repository) holds the documents and a machine-readable bibliography (CSL-JSON; a `custom` block carries `confidential`, `permission_status`, retrieval keywords, and banned aliases). An **append-only provenance ledger** (JSONL, one file per consuming repo) records every meaningful influence — which source, which decision, what claim, what kind of influence — with `cited_publicly: false` until the human flips it. And the **guardrails are mechanical**, not aspirational: the confidential entries *generate* the pattern block in each repo's untracked private-names banlist, so the existing pre-commit guard blocks the leak even when discipline fails; a `cite-guard` scan clears any text before it is shared. Later, the same machine-readable trail — sources, decisions, claims — reconstructs an academic paper, rendered publicly only through the `confidential != true` filter. +> +> "I could never let an agent near my working papers before, because one helpful footnote could burn a collaborator's trust," said Alice, a researcher-developer. "Now it reads everything, records what influenced what, and cites nothing. When Bob's paper is finally published, I flip one flag and the citation appears in the next render — with the whole influence trail already written." + +## Why This Matters + +Automatic citation is a virtue that becomes a breach the moment a source is confidential: an agent that helpfully names "the working paper this design follows" in a commit message has leaked something no history rewrite fully recalls. The naive fix — keep the material away from the agent — throws away exactly the context that makes its design work good. The resolution is to split *consultation* from *citation* and put a durable, machine-readable record between them: influence is captured eagerly and automatically (cheap, local, append-only), citation happens lazily and manually (when permission exists). The ledger is also the seed of something bigger: a paper whose claims trace to decisions and whose decisions trace to sources is *reconstructable* rather than rewritten from memory. + +This composes three existing abcd designs rather than inventing new machinery: the two-layer name banlist ([itd-74](itd-74-name-banlist.md)) supplies the leak guard; the append-only audit chain ([itd-16](itd-16-hash-chain-merkle-audit.md)) supplies the tamper-evidence model the ledger's no-remote git history approximates; and the provenance substrate ([`09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md)) already defines citation blocks, a source registry, and an NDA-aware publish gate for *ingested* content — this intent extends the same stance to *consulted* content. + +## What It Looks Like + +- **`abcd source add`** registers a document: CSL-JSON entry (confidentiality, permission status, keywords, aliases), original stored, text extracted into a grep-friendly corpus. Consultation is plain search over that corpus — no index, no service. +- **`abcd source ledger`** appends an influence record — `{decision_ref, claim, source_key, locator, influence, cited_publicly}` — and commits it; corrections are new lines, never edits. +- **`abcd source sync-banlist`** projects every confidential entry's identifying strings into the repo's untracked private-names banlist (the itd-74 private layer), so the pre-commit guard enforces what the convention promises. **`abcd source cite-check`** scans any text and reports offending entries by key only — its output is safe to relay. +- **Paper reconstruction** walks the ledger: claims grouped by decision, citations resolved from the bibliography, rendered to PDF and HTML from one markdown source; the public render draws only on the `confidential != true` filter and the human-flipped `cited_publicly` flags. + +## Dogfood (already running) + +The convention-first prototype is live for this repo's development: the corpus, ledger, guard scripts, and an agent-side consultation skill exist as a user-level scaffold, and the generated banlist block feeds the same `.githooks/pre-commit` guard that itd-74 generalises. The feature is to lift that scaffold into abcd verbs so any managed repo inherits it. See the plan ([`2026-07-08-confidential-sources-scaffold.md`](../../plans/2026-07-08-confidential-sources-scaffold.md)) and the SOTA survey ([`2026-07-08-confidential-sources-provenance-sota.md`](../../research/notes/2026-07-08-confidential-sources-provenance-sota.md)). + +## Open Questions + +- Verb shape: a `source` domain under the CLI with plugin-surface commands, or fold `ledger` into the itd-16 audit chain as a sub-verb application? +- Ledger ownership once work spans machines: per-repo files in the user-level corpus (current), or per-repo with a sync discipline? +- Whether `sync-banlist` should run automatically from the pre-commit guard itself (freshness) or stay an explicit step (surprise-free). +- How the paper pipeline's public render *proves* it consulted only permitted citations — a lint over the rendered bibliography against the ledger, or a build that structurally cannot see confidential entries. diff --git a/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md new file mode 100644 index 00000000..c4121255 --- /dev/null +++ b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md @@ -0,0 +1,79 @@ +# Plan: Confidential-sources corpus and provenance ledger (convention-first scaffold) + +> Forward-looking record: [itd-76](../intents/drafts/itd-76-source-provenance-ledger.md). +> Research grounding: [SOTA survey](../research/notes/2026-07-08-confidential-sources-provenance-sota.md). +> This plan documents the convention-first scaffold that dogfoods the intent +> before any Go code lands. + +## Context + +The user works from material they are not free to name publicly (working +papers, private repositories, NDA-shared notes). Agents need to *consult* that +material when it bears on a decision, must never *cite* it in public artifacts +(docs, commits, PR text), and every meaningful influence must be recorded in a +local-only, append-only ledger so citation remains a deliberate human act — +and so an academic paper can later be reconstructed from sources + decisions. + +Chosen shape (rejected alternatives are in the SOTA survey): one **global +corpus** in a user-level directory outside every repo (`~/.abcd/sources/`), +itself a **git repository with no remote**; **convention + skill + hooks +first**, with promotion to `abcd source` verbs recorded as itd-76. + +## The scaffold (user-level, outside this repo) + +``` +~/.abcd/sources/ + README.md the manual: schemas, workflows, guard usage + sources.json CSL-JSON bibliography — single source of truth; + custom: {confidential, permission_status, keywords, aliases, file} + corpus/.md extracted text per source, frontmatter maps hit → CSL key + files/. original documents + ledger/.jsonl append-only influence records, one file per consuming repo + bin/add-source register + convert + commit + bin/sync-banlist confidential entries → generated block in a repo's + untracked private-names banlist (the itd-74 private layer) + bin/cite-guard scan text; reports offending entries by key only +``` + +Ledger record: `{ts, repo, decision_ref, claim, source_key, locator, +influence: supports|contradicts|method|background, cited_publicly: false}` — +one JSON line per influence; corrections are new lines; `cited_publicly` flips +only by the user's hand. + +An agent-side skill (host-level, not part of this plugin) instructs the +consultation flow: grep the corpus, read matches, append a ledger line when a +source meaningfully influences a decision, never name confidential entries in +tracked or external content, and keep `sync-banlist` current so the pre-commit +guard enforces the rule mechanically. + +## Guard chain + +1. `sources.json` marks an entry `confidential: true` with `aliases`. +2. `sync-banlist` derives case-insensitive patterns (title, aliases, full + author names) and maintains a fenced generated block in the repo's + untracked `.abcd/.work.local/private-names.txt`; hand-added lines survive. +3. The committed `.githooks/pre-commit` guard (itd-74's private layer) blocks + any staged line matching — the confidential string never enters history. +4. `cite-guard` clears prose before it is shared anywhere git does not gate. + +## Verification performed (2026-07-08) + +- Fake confidential entry added; `sync-banlist` generated four patterns; a + staged file containing the fake title was **blocked** by the pre-commit + guard; removing the entry and re-syncing shrank the generated block to zero + with hand-kept lines intact. Re-running `sync-banlist` is idempotent. +- Consult flow: keyword grep over `corpus/` surfaced the seeded entry; the + frontmatter `key:` mapped the hit to its bibliography entry. +- Ledger: appended record validated with `jq -s`; the corpus repo's git log + shows the append and the later removal as separate commits (history + preserved — the tamper-evidence property). +- `cite-guard`: exit 1 with key-only report on offending text; exit 0 clean. + +## Out of scope (recorded, not built) + +- **Paper pipeline**: Quarto, two project profiles (internal/public); public + render draws only on the `confidential != true` bibliography filter plus + human-flipped `cited_publicly` flags. Build when the ledger has real entries. +- **`abcd source` verbs** (add / consult / ledger / cite-check): itd-76. +- **Retrieval upgrades** (SQLite FTS5, embeddings): only if grep over the + corpus gets noisy; the survey records why grep is the default. diff --git a/.abcd/development/principles/personas-alice-bob-carol.md b/.abcd/development/principles/personas-alice-bob-carol.md new file mode 100644 index 00000000..9276e0fd --- /dev/null +++ b/.abcd/development/principles/personas-alice-bob-carol.md @@ -0,0 +1,20 @@ +# Personas are always Alice, Bob, Carol + +Every persona in a user story, press-release quote, worked example, or scenario +is named from the fixed sequence **Alice, Bob, Carol, …** — never any other +invented name, never a real person's name as a stand-in. A user story begins +with Alice; a second role is Bob; a third, Carol. + +The real user, when referred to at all, is **they/them** — never he or she. + +**Why.** Persona names are load-bearing noise: a bespoke name per document +("Dev, a maintainer") makes readers wonder whether the name carries meaning, +and a real-sounding name risks colliding with an actual person or project. The +Alice/Bob convention is instantly recognisable as "generic actor, role by +position", costs nothing to apply, and makes cross-document reading uniform — +Alice is always the primary actor, Bob always the counterpart. + +**Applies to.** Intents (press-release quotes and scenarios), brief surface +docs, ADR examples, plans, and any user-facing prose that stages a scenario. +Existing documents that predate this principle are corrected opportunistically +when otherwise edited, not swept. diff --git a/.abcd/development/research/_references.md b/.abcd/development/research/_references.md index 1638cddb..5a1efd51 100644 --- a/.abcd/development/research/_references.md +++ b/.abcd/development/research/_references.md @@ -46,6 +46,30 @@ Canonical bibliography for `.abcd/development/` documents. When citing prior art [amazon-working-backwards]: https://www.allthingsdistributed.com/2006/11/working_backwards.html "Working Backwards (Vogels) — Amazon press-release-first product design" ``` +### Citation, provenance, and publishing + +``` +[csl-spec]: https://docs.citationstyles.org/en/stable/specification.html "Citation Style Language 1.0.2 specification" +[csl-schema]: https://github.com/citation-style-language/schema/blob/master/schemas/input/csl-data.json "CSL-JSON input schema (csl-data.json)" +[pandoc]: https://pandoc.org/MANUAL.html "Pandoc user's guide — citeproc and bibliography formats" +[better-bibtex]: https://retorque.re/zotero-better-bibtex/exporting/extra-fields/ "Better BibTeX for Zotero — extra fields and custom-field round-tripping" +[hayagriva]: https://github.com/typst/hayagriva "Hayagriva — Typst bibliography file format" +[prov-json]: https://www.w3.org/submissions/prov-json/ "PROV-JSON — W3C member submission" +[adr-org]: https://adr.github.io/ "Architectural Decision Records — homepage and conventions" +[agent-evidence]: https://arxiv.org/pdf/2606.04990 "Survey of evidence tracing and provenance in LLM agents (arXiv 2606.04990)" +[finos-mi13]: https://air-governance-framework.finos.org/mitigations/mi-13_providing-citations-and-source-traceability-for-ai-generated-information.html "FINOS AI governance framework — MI-13 citations and source traceability" +[grep-vs-embeddings]: https://jxnl.co/writing/2025/09/11/why-grep-beat-embeddings-in-our-swe-bench-agent-lessons-from-augment/ "Why grep beat embeddings in a SWE-bench agent (Liu / Augment)" +[no-indexing]: https://vadim.blog/claude-code-no-indexing/ "Analysis of a major coding agent's grep-over-index retrieval design" +[grep-nuance]: https://www.nuss-and-bolts.com/p/on-the-lost-nuance-of-grep-vs-semantic "On the lost nuance of grep vs semantic search" +[sqlite-vec]: https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/index.html "Hybrid full-text + vector search in SQLite (Garcia)" +[gitleaks-precommit]: https://m3ssap0.github.io/2023/09/29/pre-commit-gitleaks.html "Custom gitleaks rules in a pre-commit hook" +[vale]: https://vale.sh/ "Vale — syntax-aware prose linter" +[quarto-profiles]: https://quarto.org/docs/projects/profiles.html "Quarto project profiles" +[quarto-conditional]: https://quarto.org/docs/authoring/conditional.html "Quarto conditional content" +[quarto-typst]: https://quarto.org/docs/output-formats/typst.html "Quarto Typst PDF output" +[manubot]: https://manubot.org/ "Manubot — git-based manuscripts with automated citation resolution" +``` + ## Adding a new reference 1. Pick a slug — lowercase kebab, ≤ 30 chars, project-name-shaped. diff --git a/.abcd/development/research/notes/2026-07-08-confidential-sources-provenance-sota.md b/.abcd/development/research/notes/2026-07-08-confidential-sources-provenance-sota.md new file mode 100644 index 00000000..bfabd020 --- /dev/null +++ b/.abcd/development/research/notes/2026-07-08-confidential-sources-provenance-sota.md @@ -0,0 +1,141 @@ +# SOTA survey: confidential source consultation, provenance ledgers, and paper reconstruction + +Grounding for [itd-76](../../intents/drafts/itd-76-source-provenance-ledger.md) +and the [convention-first scaffold plan](../../plans/2026-07-08-confidential-sources-scaffold.md). +Question: what is the best current (2024–2026) shape for a **local-only** +system in which an agent may *consult* confidential documents, never *cites* +them automatically, records source→decision influence in an append-only +ledger, and later reconstructs an academic paper (PDF + HTML) from that trail? + +Five sub-questions, each with the ranked finding and the evidence tier +(official spec / widely adopted practice / practitioner anecdote). + +## 1. Bibliography format — CSL-JSON wins + +**Chosen: CSL-JSON as the single source of truth; `.bib` generated on demand.** + +- CSL-JSON's reserved `custom` field is *specified* for exactly this job: + key–value data outside the citation model, ignored by citeproc during + rendering — so `confidential`, `permission_status`, `keywords`, `aliases` + ride along without ever surfacing in formatted output ([CSL 1.0.2 + spec][csl-spec], [input schema][csl-schema]; official spec). Pandoc consumes + CSL-JSON natively ([manual][pandoc]; official docs), and it is Zotero's + native export. +- BibTeX/BibLaTeX tolerates unknown fields but has no schema; custom-field + semantics are exporter-specific ([Better BibTeX extra fields][better-bibtex]; + adopted practice). Keep it as a generated artifact for LaTeX toolchains. +- Hayagriva (Typst's YAML format) defines a closed field list with no + sanctioned custom-field mechanism ([format spec][hayagriva]; official docs) + — no home for a `confidential` flag; Typst reads `.bib` anyway. +- A public bibliography is one filter: `custom.confidential != true`. + +## 2. Provenance ledger — JSONL with PROV-shaped vocabulary + +**Chosen: append-only JSONL, one record per influence; ADRs reference ledger +entries, never the reverse.** + +- No de-facto standard exists; 2025–26 agent-provenance work converges on + append-only JSONL event logs ([evidence-tracing survey][agent-evidence]; + practitioner consensus). RAG audit practice contributes the load-bearing + linkage — claim ↔ source ↔ locator — that makes later paper reconstruction + possible ([FINOS AI governance MI-13][finos-mi13]; industry guidance). +- W3C PROV is the only true standard ([PROV-JSON][prov-json]) but its + Entity/Activity/Agent ceremony has no consumer at solo-developer scale — + *borrow the vocabulary* (`influence`, derivation kinds) so a later export is + mechanical; do not adopt the format. +- ADR practice is already append-only ("supersede and link, never edit" — + [adr.github.io][adr-org]; adopted practice), matching this repo's + `DECISIONS.md` convention: the ledger is the machine layer beneath it. + +## 3. Local consultation — grep over a converted corpus; no RAG + +**Chosen: extract every document to markdown/text once (pandoc / pdftotext); +agents search it with plain grep. SQLite FTS5 is the recorded upgrade path; +vector RAG is rejected at this scale.** + +- The strongest-evidenced finding of the survey: agentic grep displaced + embedding indexes in major coding agents in 2025, with direct benchmark + support ([why grep beat embeddings on a SWE-bench agent][grep-vs-embeddings]; + [analysis of the no-indexing design][no-indexing]; adopted practice). + A corpus of tens-to-hundreds of documents fits the paradigm trivially. +- Mitigation for prose (where vocabulary mismatch bites harder than in code): + grep hand-written `keywords:` frontmatter, author names, and citation keys — + not only content strings (practitioner nuance, [grep-vs-semantic + caveat][grep-nuance]). +- If recall degrades: single-file SQLite FTS5 (BM25), optionally hybrid with + local vectors ([sqlite-vec hybrid search][sqlite-vec]; practitioner + anecdote). Full local RAG stacks add indexing pipelines and failure modes a + grep answers in milliseconds — overkill (consensus across the surveyed + write-ups). + +## 4. Leakage guardrails — generate the denylist from the bibliography + +**Chosen: structural separation (confidential strings exist only in the +user-level corpus and untracked banlists) plus mechanical scanning derived +from the marked entries.** + +- The two-layer split this repo already runs (public banned tokens in CI lint; + private patterns in an untracked pre-commit banlist, because a public rule + cannot contain the secret it bans) is the right architecture — + [itd-74](../../intents/drafts/itd-74-name-banlist.md) generalises it. +- The new move: *derive* the private patterns from the bibliography's + `confidential: true` entries (title, aliases, full author names) so the + banlist cannot drift from the corpus. Custom-rule secret scanners at + pre-commit and CI are the established analogue ([gitleaks custom rules at + pre-commit][gitleaks-precommit]; adopted practice). Prose linters with + forbidden-term rules serve the same role in docs pipelines ([Vale][vale]; + adopted practice) — this repo's docs-lint banned-token family already covers + that surface. +- Hooks defend against honest mistakes, not bypass; this repo already bans + `--no-verify`. No off-the-shelf taint tracking for LLM output exists — the + ledger's `cited_publicly` flag plus a release-time check (no rendered + citation whose entry forbids it) approximates it. + +## 5. Paper reconstruction — Quarto + +**Chosen: Quarto, one markdown source, dual render (PDF via bundled Typst; +HTML), citeproc over the CSL-JSON.** + +- Quarto is the only candidate hitting every requirement natively, and — + decisive here — **project profiles + conditional content** give a + *structural* citation firewall: the public profile renders from the filtered + bibliography and simply cannot see confidential entries + ([profiles][quarto-profiles], [conditional content][quarto-conditional], + [Typst output][quarto-typst]; official docs). Install weight: one binary. +- Bare Pandoc + Makefile is the same engine minus the profile machinery — + acceptable fallback, more glue. +- Typst alone: no real HTML story, closed bibliography schema (§1). Manubot's + cite-by-public-identifier model is structurally wrong for sources that have + no public identifier ([manubot][manubot]; official docs) — anti-recommended + for this use case. + +## Synthesis + +CSL-JSON bibliography (+ `custom` block) → grep-consulted converted corpus → +append-only JSONL ledger keyed to decisions → banlist patterns generated from +the confidential entries into the itd-74 private guard layer → Quarto +public/internal profiles for eventual PDF + HTML. Nothing needs a service, +daemon, or index; the two bespoke pieces (ledger schema, release-time citation +check) are deliberately tiny because the field genuinely has no standard there. + +## References + +[csl-spec]: https://docs.citationstyles.org/en/stable/specification.html "Citation Style Language 1.0.2 specification" +[csl-schema]: https://github.com/citation-style-language/schema/blob/master/schemas/input/csl-data.json "CSL-JSON input schema (csl-data.json)" +[pandoc]: https://pandoc.org/MANUAL.html "Pandoc user's guide — citeproc and bibliography formats" +[better-bibtex]: https://retorque.re/zotero-better-bibtex/exporting/extra-fields/ "Better BibTeX for Zotero — extra fields and custom-field round-tripping" +[hayagriva]: https://github.com/typst/hayagriva "Hayagriva — Typst bibliography file format" +[prov-json]: https://www.w3.org/submissions/prov-json/ "PROV-JSON — W3C member submission" +[adr-org]: https://adr.github.io/ "Architectural Decision Records — homepage and conventions" +[agent-evidence]: https://arxiv.org/pdf/2606.04990 "Survey of evidence tracing and provenance in LLM agents (arXiv 2606.04990)" +[finos-mi13]: https://air-governance-framework.finos.org/mitigations/mi-13_providing-citations-and-source-traceability-for-ai-generated-information.html "FINOS AI governance framework — MI-13 citations and source traceability" +[grep-vs-embeddings]: https://jxnl.co/writing/2025/09/11/why-grep-beat-embeddings-in-our-swe-bench-agent-lessons-from-augment/ "Why grep beat embeddings in a SWE-bench agent (Liu / Augment)" +[no-indexing]: https://vadim.blog/claude-code-no-indexing/ "Analysis of a major coding agent's grep-over-index retrieval design" +[grep-nuance]: https://www.nuss-and-bolts.com/p/on-the-lost-nuance-of-grep-vs-semantic "On the lost nuance of grep vs semantic search" +[sqlite-vec]: https://alexgarcia.xyz/blog/2024/sqlite-vec-hybrid-search/index.html "Hybrid full-text + vector search in SQLite (Garcia)" +[gitleaks-precommit]: https://m3ssap0.github.io/2023/09/29/pre-commit-gitleaks.html "Custom gitleaks rules in a pre-commit hook" +[vale]: https://vale.sh/ "Vale — syntax-aware prose linter" +[quarto-profiles]: https://quarto.org/docs/projects/profiles.html "Quarto project profiles" +[quarto-conditional]: https://quarto.org/docs/authoring/conditional.html "Quarto conditional content" +[quarto-typst]: https://quarto.org/docs/output-formats/typst.html "Quarto Typst PDF output" +[manubot]: https://manubot.org/ "Manubot — git-based manuscripts with automated citation resolution" diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index 26b71b18..3a88b2f1 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -21,3 +21,10 @@ parallel-agent merge contention bites. .work.local (local). `docs/` user-facing only. - 2026-07-06 — Module path `github.com/REPPL/abcd-cli`; Cobra approved as the CLI framework (matches ferry and the companion harness). +- 2026-07-08 — Confidential sources: global user-level corpus (CSL-JSON + grep + corpus, local no-remote git), append-only JSONL influence ledger per repo, + banlist patterns generated from confidential entries into the itd-74 private + guard; convention + skill first, `abcd source` verbs deferred (itd-76). Quarto + chosen for eventual paper reconstruction; RAG rejected at this scale. +- 2026-07-08 — Personas in any scenario are always Alice, Bob, Carol (in that + order); the user is they/them. Recorded as a principle. From 01a3b1b44494178d167f7da86bf8b3f0334215c0 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Wed, 8 Jul 2026 12:56:30 +0100 Subject: [PATCH 02/11] docs: harden itd-76 after design grill; add itd-77 (relocatable user home) Grill outcomes: leak guard promises literal strings only (paraphrase risk stated and handled behaviourally); two-level AND citation gate; author bans default on with per-source ban_authors opt-out; standalone source domain (itd-16 optional backend); public render proven by structural filter plus post-render lint; team sharing of citation data via committed .abcd/work/references.json (share/ingest); durability via machine backup + git bundle. The pre-commit guard now auto-refreshes the generated banlist block from the user-level corpus when present. itd-77 records the user-level ~/.abcd home as a configurable, wizard- relocatable fourth tier, additive to the repo .abcd tiers. Assisted-by: Claude:claude-fable-5 --- .../drafts/itd-76-source-provenance-ledger.md | 28 +++++----- .../drafts/itd-77-relocatable-user-home.md | 33 ++++++++++++ ...026-07-08-confidential-sources-scaffold.md | 51 +++++++++++++++---- .abcd/work/DECISIONS.md | 12 +++++ .githooks/pre-commit | 11 ++++ 5 files changed, 113 insertions(+), 22 deletions(-) create mode 100644 .abcd/development/intents/drafts/itd-77-relocatable-user-home.md diff --git a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md index 1c073031..507045d8 100644 --- a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md +++ b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md @@ -11,22 +11,27 @@ reclassification_history: [] ## Press Release -> **Consult any source freely; cite only by deliberate human choice — abcd keeps the ledger in between.** A developer-researcher often works from material they are not free to name in public: working papers under review, a collaborator's private repository, notes shared under an NDA. The agent should be able to *read* that material when it bears on a decision — and must never *cite* it in anything published. abcd manages this as three coupled pieces. A **local-only corpus** (a user-level directory outside every repo, itself a no-remote git repository) holds the documents and a machine-readable bibliography (CSL-JSON; a `custom` block carries `confidential`, `permission_status`, retrieval keywords, and banned aliases). An **append-only provenance ledger** (JSONL, one file per consuming repo) records every meaningful influence — which source, which decision, what claim, what kind of influence — with `cited_publicly: false` until the human flips it. And the **guardrails are mechanical**, not aspirational: the confidential entries *generate* the pattern block in each repo's untracked private-names banlist, so the existing pre-commit guard blocks the leak even when discipline fails; a `cite-guard` scan clears any text before it is shared. Later, the same machine-readable trail — sources, decisions, claims — reconstructs an academic paper, rendered publicly only through the `confidential != true` filter. +> **Consult any source freely; cite only by deliberate human choice — abcd keeps the ledger in between.** A developer-researcher often works from material they are not free to name in public: working papers under review, a collaborator's private repository, notes shared under an NDA. The agent should be able to *read* that material when it bears on a decision — and must never *cite* it in anything published. abcd manages this as three coupled pieces. A **local-only corpus** (abcd's user-level home, `~/.abcd/sources/` by default, itself a no-remote git repository) holds the documents and a machine-readable bibliography (CSL-JSON; a `custom` block carries `confidential`, `permission_status`, retrieval keywords, and banned aliases). An **append-only provenance ledger** (JSONL, one file per consuming repo) records every meaningful influence — which source, which decision, what claim, what kind of influence — gated twice: the source's `permission_status` grants the *right* to cite, and each ledger line's `cited_publicly` flag *exercises* it, flipped only by the human. And the **guardrails are mechanical for what mechanism can catch**: the confidential entries *generate* the pattern block in each repo's untracked private-names banlist, the pre-commit guard refreshes and enforces it on every commit, and a `cite-guard` scan clears any text before it is shared. Published sources travel the other way: their *citation data* (never the documents, never the ledger) is shared into the repo, so a whole team can ingest and build on one bibliography. Later, the same machine-readable trail — sources, decisions, claims — reconstructs an academic paper, publicly rendered only through a structurally filtered bibliography and a post-render check. > -> "I could never let an agent near my working papers before, because one helpful footnote could burn a collaborator's trust," said Alice, a researcher-developer. "Now it reads everything, records what influenced what, and cites nothing. When Bob's paper is finally published, I flip one flag and the citation appears in the next render — with the whole influence trail already written." +> "I could never let an agent near my working papers before, because one helpful footnote could burn a collaborator's trust," said Alice, a researcher-developer. "Now it reads everything, records what influenced what, and cites nothing. When the paper behind a decision is finally published, I flip one flag and the citation appears in the next render — with the whole influence trail already written." Her teammate Bob never sees her corpus at all: "I just ingest the repo's shared references and every public source Alice worked from is in my own local store, ready to consult." ## Why This Matters -Automatic citation is a virtue that becomes a breach the moment a source is confidential: an agent that helpfully names "the working paper this design follows" in a commit message has leaked something no history rewrite fully recalls. The naive fix — keep the material away from the agent — throws away exactly the context that makes its design work good. The resolution is to split *consultation* from *citation* and put a durable, machine-readable record between them: influence is captured eagerly and automatically (cheap, local, append-only), citation happens lazily and manually (when permission exists). The ledger is also the seed of something bigger: a paper whose claims trace to decisions and whose decisions trace to sources is *reconstructable* rather than rewritten from memory. +Automatic citation is a virtue that becomes a breach the moment a source is confidential: an agent that helpfully names "the working paper this design follows" in a commit message has leaked something no history rewrite fully recalls. The naive fix — keep the material away from the agent — throws away exactly the context that makes its design work good. The resolution is to split *consultation* from *citation* and put a durable, machine-readable record between them: influence is captured eagerly and automatically (cheap, local, append-only), citation happens lazily and manually (when permission exists). The ledger is also the seed of something bigger: a paper whose claims trace to decisions and whose decisions trace to sources is *reconstructable* rather than rewritten from memory. And because a repo is a team surface even when every corpus is personal, the public slice of the bibliography must flow through the repo — citation data is shareable; documents and influence trails are not. -This composes three existing abcd designs rather than inventing new machinery: the two-layer name banlist ([itd-74](itd-74-name-banlist.md)) supplies the leak guard; the append-only audit chain ([itd-16](itd-16-hash-chain-merkle-audit.md)) supplies the tamper-evidence model the ledger's no-remote git history approximates; and the provenance substrate ([`09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md)) already defines citation blocks, a source registry, and an NDA-aware publish gate for *ingested* content — this intent extends the same stance to *consulted* content. +This composes three existing abcd designs rather than inventing new machinery: the two-layer name banlist ([itd-74](itd-74-name-banlist.md)) supplies the leak guard; the append-only audit chain ([itd-16](itd-16-hash-chain-merkle-audit.md)) is a possible later integrity backend for the ledger — the corpus repo's git history carries tamper-evidence until then, and nothing here depends on itd-16 shipping; and the provenance substrate ([`09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md)) already defines citation blocks, a source registry, and an NDA-aware publish gate for *ingested* content — this intent extends the same stance to *consulted* content. ## What It Looks Like -- **`abcd source add`** registers a document: CSL-JSON entry (confidentiality, permission status, keywords, aliases), original stored, text extracted into a grep-friendly corpus. Consultation is plain search over that corpus — no index, no service. -- **`abcd source ledger`** appends an influence record — `{decision_ref, claim, source_key, locator, influence, cited_publicly}` — and commits it; corrections are new lines, never edits. -- **`abcd source sync-banlist`** projects every confidential entry's identifying strings into the repo's untracked private-names banlist (the itd-74 private layer), so the pre-commit guard enforces what the convention promises. **`abcd source cite-check`** scans any text and reports offending entries by key only — its output is safe to relay. -- **Paper reconstruction** walks the ledger: claims grouped by decision, citations resolved from the bibliography, rendered to PDF and HTML from one markdown source; the public render draws only on the `confidential != true` filter and the human-flipped `cited_publicly` flags. +- **`abcd source add`** registers a document: CSL-JSON entry (confidentiality, permission status, keywords, aliases), original stored, text extracted into a grep-friendly corpus. Consultation is plain search over that corpus — no index, no service. The corpus lives in abcd's **user-level home** (`~/.abcd/`, path configurable; relocation is [itd-77](itd-77-relocatable-user-home.md)) — the first user-tier surface alongside the repo's `.abcd/` tiers. +- **`abcd source ledger`** appends an influence record — `{decision_ref, claim, source_key, locator, influence, cited_publicly}` — and commits it; corrections are new lines, never edits. A public citation requires **both** gates: the source permits (`permission_status`) *and* the line is flipped (`cited_publicly: true`). +- **`abcd source share` / `abcd source ingest`** move *citation data* through the repo: `share` writes a public entry into the committed `.abcd/work/references.json` (refusing any `confidential: true` entry mechanically); `ingest` imports the repo's shared entries into the local corpus. Documents and ledgers never travel — only bibliography. The hand-curated references registry in the development record can later derive from this file. +- **`abcd source sync-banlist`** projects every confidential entry's identifying strings — title, aliases, and author names (author bans default on, per-source opt-out via `ban_authors: false` for authors with citable public work) — into the repo's untracked private-names banlist (the itd-74 private layer). The pre-commit guard **auto-refreshes** this block when the corpus is present (no-op otherwise), so the guard is only ever as stale as the current commit. **`abcd source cite-check`** scans any text and reports offending entries by key only — its output is safe to relay. +- **Paper reconstruction** walks the ledger: claims grouped by decision, citations resolved from the bibliography, rendered to PDF and HTML from one markdown source. The public render is proven clean twice, independently: structurally (it renders from a *generated* bibliography that omits confidential entries, so an unpermitted key fails the build) and by a deterministic post-render check of the output's citations against both gates. + +## What It Cannot Enforce + +The mechanical layer blocks **literal identifying strings**. It cannot detect a paraphrase that identifies a source without naming it — "a forthcoming paper shows X beats Y", or a description of a private repository's distinctive architecture. That residual risk is handled behaviourally (the consultation skill forbids identifying description, not just naming) and by the human review that gates every publish; abcd states this boundary plainly rather than implying coverage it does not have. Durability is likewise bounded: a no-remote corpus survives disk loss only via machine backup and offline `git bundle` snapshots — abcd documents that discipline; it cannot perform it. ## Dogfood (already running) @@ -34,7 +39,6 @@ The convention-first prototype is live for this repo's development: the corpus, ## Open Questions -- Verb shape: a `source` domain under the CLI with plugin-surface commands, or fold `ledger` into the itd-16 audit chain as a sub-verb application? -- Ledger ownership once work spans machines: per-repo files in the user-level corpus (current), or per-repo with a sync discipline? -- Whether `sync-banlist` should run automatically from the pre-commit guard itself (freshness) or stay an explicit step (surprise-free). -- How the paper pipeline's public render *proves* it consulted only permitted citations — a lint over the rendered bibliography against the ledger, or a build that structurally cannot see confidential entries. +- Ledger ownership once work spans machines: per-repo files in the user-level corpus (current) work for one machine; deferred until a second machine actually exists. +- `share`/`ingest` conflict shape when two teammates share the same source with divergent metadata — last-write, merge, or key-ownership. +- Whether ingest should mark provenance on imported entries (who shared, from which repo) so a team bibliography stays auditable. diff --git a/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md b/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md new file mode 100644 index 00000000..eeca4c25 --- /dev/null +++ b/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md @@ -0,0 +1,33 @@ +--- +id: itd-77 +slug: relocatable-user-home +spec_id: null +kind: standalone +suggested_kind: null +reclassification_history: [] +--- + +# abcd's User-Level Home Lives Where You Want It + +## Press Release + +> **One user-level home for everything abcd keeps outside your repos — blessed by default, movable by choice.** abcd's repo-tree `.abcd/` tiers cover what belongs to a project. But some state belongs to the *person*: the confidential-sources corpus ([itd-76](itd-76-source-provenance-ledger.md)), and whatever user-scoped configuration follows it. That state lives in abcd's user-level home — `~/.abcd/` by default, the same dotfile convention as every tool a developer already trusts with home-directory state. The default is only a default: a guided, wizard-style flow moves the home wherever the user's filing system wants it — say, alongside their projects root — updates the recorded location, and verifies every consumer (guards, skills, verbs) resolves the new path before declaring the move done. The user tier is *additive*: repo `.abcd/` tiers are untouched, and a repo remains fully functional on machines where no user home exists at all. +> +> "My whole development world lives under one directory, and I wanted abcd's user state in there too, not scattered in my home directory," said Alice. "One command moved it, re-pointed everything that reads it, and proved the pre-commit guard still found my banlist before it called the move complete." + +## Why This Matters + +The moment abcd grew a user-level surface, it inherited a question every long-lived tool answers eventually: whose filing system wins, the tool's or the user's? Hard-coding `~/.abcd/` answers "the tool's" forever; making the path a per-callsite option answers "nobody's" and scatters resolution logic across every consumer. The right shape is a single blessed default plus a single recorded override that every consumer resolves through one function — cheap now, and it prevents the class of bug where one consumer honours the moved home and another silently recreates the old one. A rehearsed, verified move matters more than the move itself: user state includes material whose guards must not lapse mid-relocation (itd-76's banlist source), so the flow proves consumers resolve the new location before the old one is retired. + +## What It Looks Like + +- **One resolution rule.** Every abcd component that touches user-level state resolves the home through a single lookup: explicit config, else environment override, else `~/.abcd/`. No consumer hard-codes the path. +- **A guided move.** An interactive, wizard-style flow relocates the home: copies (never moves-then-prays) the tree to the target, records the new location, re-runs each consumer's own verification (the itd-76 guard finding its banlist source, skills resolving the corpus), and only then retires the old tree — a rehearsed cutover with a rollback path, not a rename. +- **Additive to repo tiers.** The user home complements the repo's `.abcd/` layout; nothing repo-side moves, and every repo continues to work on machines with no user home (guards no-op exactly as the itd-74 banlist does on fresh clones). +- **Doctor-visible.** The install-health check reports where the user home resolves and flags a dangling override (recorded path that no longer exists). + +## Open Questions + +- Where the override is recorded: a user-level config file at a fixed bootstrap location (a pointer must live *somewhere* unmovable), an environment variable, or both with defined precedence. +- Whether repo-level config may pin a user-home path for reproducibility, or whether that inverts the tiers (a repo dictating personal filing) and should be refused. +- Migration of stale references after a move — e.g. per-repo banlists regenerated on next commit by the itd-76 auto-refresh, or swept eagerly by the move flow. diff --git a/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md index c4121255..807f2c10 100644 --- a/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md +++ b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md @@ -38,7 +38,9 @@ first**, with promotion to `abcd source` verbs recorded as itd-76. Ledger record: `{ts, repo, decision_ref, claim, source_key, locator, influence: supports|contradicts|method|background, cited_publicly: false}` — one JSON line per influence; corrections are new lines; `cited_publicly` flips -only by the user's hand. +only by the user's hand. Public citation is a **two-level AND**: the source's +`permission_status` grants the right; the line's `cited_publicly` exercises it +per claim. An agent-side skill (host-level, not part of this plugin) instructs the consultation flow: grep the corpus, read matches, append a ledger line when a @@ -49,13 +51,36 @@ guard enforces the rule mechanically. ## Guard chain 1. `sources.json` marks an entry `confidential: true` with `aliases`. -2. `sync-banlist` derives case-insensitive patterns (title, aliases, full - author names) and maintains a fenced generated block in the repo's - untracked `.abcd/.work.local/private-names.txt`; hand-added lines survive. -3. The committed `.githooks/pre-commit` guard (itd-74's private layer) blocks - any staged line matching — the confidential string never enters history. +2. `sync-banlist` derives case-insensitive patterns (title, aliases, and full + author names — author bans default on, per-source opt-out via + `custom.ban_authors: false` for authors with citable public work) and + maintains a fenced generated block in the repo's untracked + `.abcd/.work.local/private-names.txt`; hand-added lines survive. +3. The committed `.githooks/pre-commit` guard (itd-74's private layer) + **auto-refreshes** the generated block when the corpus is present (no-op + otherwise — CI, fresh clones), then blocks any staged line matching — the + confidential string never enters history and the guard is never stale. 4. `cite-guard` clears prose before it is shared anywhere git does not gate. +**Boundary (stated, not hidden):** the mechanical layer blocks literal +identifying strings only. Paraphrase that identifies a source without naming +it is handled behaviourally (the skill forbids identifying description) and by +human review of every publish. Durability: the no-remote corpus rides the +machine backup, plus occasional `git bundle` snapshots to an encrypted +external volume; multi-machine ledger ownership is deferred until a second +machine exists. + +## Team sharing (citation data only) + +A repo is a team surface even when every corpus is personal. Public entries' +*citation data* — never documents, never ledgers — flow through the committed +`.abcd/work/references.json` (CSL-JSON): `share` writes an entry there, +refusing `confidential: true` mechanically; `ingest` imports the repo's shared +entries into a teammate's local corpus. Recorded in itd-76; scripts are built +when a second contributor exists. The hand-curated +`development/research/_references.md` registry stays human-owned and may later +derive from this file. + ## Verification performed (2026-07-08) - Fake confidential entry added; `sync-banlist` generated four patterns; a @@ -71,9 +96,15 @@ guard enforces the rule mechanically. ## Out of scope (recorded, not built) -- **Paper pipeline**: Quarto, two project profiles (internal/public); public - render draws only on the `confidential != true` bibliography filter plus - human-flipped `cited_publicly` flags. Build when the ledger has real entries. -- **`abcd source` verbs** (add / consult / ledger / cite-check): itd-76. +- **Paper pipeline**: Quarto, two project profiles (internal/public); the + public render is proven clean twice — structurally (it renders from a + generated bibliography that omits confidential entries, so an unpermitted + key fails the build) and by a deterministic post-render check of the + output's citations against both gates. Build when the ledger has real + entries. +- **`abcd source` verbs** (add / consult / ledger / share / ingest / + cite-check): itd-76, as a standalone core domain (itd-16's audit chain is a + possible later ledger backend, not a dependency). The user-level home it + reads is itd-77's surface. - **Retrieval upgrades** (SQLite FTS5, embeddings): only if grep over the corpus gets noisy; the survey records why grep is the default. diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index 3a88b2f1..d61102a9 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -28,3 +28,15 @@ parallel-agent merge contention bites. chosen for eventual paper reconstruction; RAG rejected at this scale. - 2026-07-08 — Personas in any scenario are always Alice, Bob, Carol (in that order); the user is they/them. Recorded as a principle. +- 2026-07-08 — itd-76 grilled: leak guard promises literal strings only + (paraphrase risk stated, handled behaviourally + review); citation is a + two-level AND (source permission_status AND per-line cited_publicly); author + bans default on with per-source ban_authors opt-out; standalone `source` + domain (itd-16 a possible backend, not a dependency); pre-commit auto- + refreshes the generated banlist; public render proven by structural filter + AND post-render lint; team share of citation data via committed + `.abcd/work/references.json` (share/ingest); durability = machine backup + + git bundle, multi-machine deferred. +- 2026-07-08 — `~/.abcd/` blessed as abcd's user-level home (fourth tier, + additive to repo `.abcd/`), path configurable; relocation wizard recorded as + itd-77. diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 6ec030b7..3389df54 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -17,6 +17,17 @@ set -euo pipefail cd "$(git rev-parse --show-toplevel)" banlist=".abcd/.work.local/private-names.txt" + +# Refresh the generated block from the user-level sources corpus when present +# (itd-76 dogfood): keeps the banlist as fresh as the current commit. No-op on +# machines without the corpus; a failed refresh falls back to the existing +# (possibly stale) banlist with a warning rather than blocking the commit. +sync_banlist="$HOME/.abcd/sources/bin/sync-banlist" +if [ -x "$sync_banlist" ]; then + "$sync_banlist" "$(pwd)" >/dev/null 2>&1 \ + || echo "pre-commit: warning — banlist refresh failed; checking existing banlist." >&2 +fi + [ -f "$banlist" ] || exit 0 # Only added lines of staged content matter (skip the +++ file headers). From 56d0ca3eb96f712017be18f0298e25a881ec830b Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Wed, 8 Jul 2026 13:04:41 +0100 Subject: [PATCH 03/11] docs: flip author bans to opt-in (ban_authors: true) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Supersedes today's default-on choice with base-rate information: the expected corpus (own submitted work, purchased reports, private repos) makes default author bans near-pure false positives — they would ban the user's own name — while title/alias patterns carry the actual protection. Author bans remain available per source for collaborations that are themselves secret. Assisted-by: Claude:claude-fable-5 --- .../drafts/itd-76-source-provenance-ledger.md | 2 +- .../2026-07-08-confidential-sources-scaffold.md | 13 ++++++++----- .abcd/work/DECISIONS.md | 5 +++++ 3 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md index 507045d8..ac7834aa 100644 --- a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md +++ b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md @@ -26,7 +26,7 @@ This composes three existing abcd designs rather than inventing new machinery: t - **`abcd source add`** registers a document: CSL-JSON entry (confidentiality, permission status, keywords, aliases), original stored, text extracted into a grep-friendly corpus. Consultation is plain search over that corpus — no index, no service. The corpus lives in abcd's **user-level home** (`~/.abcd/`, path configurable; relocation is [itd-77](itd-77-relocatable-user-home.md)) — the first user-tier surface alongside the repo's `.abcd/` tiers. - **`abcd source ledger`** appends an influence record — `{decision_ref, claim, source_key, locator, influence, cited_publicly}` — and commits it; corrections are new lines, never edits. A public citation requires **both** gates: the source permits (`permission_status`) *and* the line is flipped (`cited_publicly: true`). - **`abcd source share` / `abcd source ingest`** move *citation data* through the repo: `share` writes a public entry into the committed `.abcd/work/references.json` (refusing any `confidential: true` entry mechanically); `ingest` imports the repo's shared entries into the local corpus. Documents and ledgers never travel — only bibliography. The hand-curated references registry in the development record can later derive from this file. -- **`abcd source sync-banlist`** projects every confidential entry's identifying strings — title, aliases, and author names (author bans default on, per-source opt-out via `ban_authors: false` for authors with citable public work) — into the repo's untracked private-names banlist (the itd-74 private layer). The pre-commit guard **auto-refreshes** this block when the corpus is present (no-op otherwise), so the guard is only ever as stale as the current commit. **`abcd source cite-check`** scans any text and reports offending entries by key only — its output is safe to relay. +- **`abcd source sync-banlist`** projects every confidential entry's identifying strings — title and aliases always; author names only by per-source opt-in (`ban_authors: true`, for the rare collaboration that is itself secret — the common confidential types, one's own submitted work, purchased reports, and private repos, are protected by title and aliases, and banning their authors would mostly ban legitimate names, including one's own) — into the repo's untracked private-names banlist (the itd-74 private layer). The pre-commit guard **auto-refreshes** this block when the corpus is present (no-op otherwise), so the guard is only ever as stale as the current commit. **`abcd source cite-check`** scans any text and reports offending entries by key only — its output is safe to relay. - **Paper reconstruction** walks the ledger: claims grouped by decision, citations resolved from the bibliography, rendered to PDF and HTML from one markdown source. The public render is proven clean twice, independently: structurally (it renders from a *generated* bibliography that omits confidential entries, so an unpermitted key fails the build) and by a deterministic post-render check of the output's citations against both gates. ## What It Cannot Enforce diff --git a/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md index 807f2c10..4e7db096 100644 --- a/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md +++ b/.abcd/development/plans/2026-07-08-confidential-sources-scaffold.md @@ -51,11 +51,14 @@ guard enforces the rule mechanically. ## Guard chain 1. `sources.json` marks an entry `confidential: true` with `aliases`. -2. `sync-banlist` derives case-insensitive patterns (title, aliases, and full - author names — author bans default on, per-source opt-out via - `custom.ban_authors: false` for authors with citable public work) and - maintains a fenced generated block in the repo's untracked - `.abcd/.work.local/private-names.txt`; hand-added lines survive. +2. `sync-banlist` derives case-insensitive patterns — title and aliases + always; full author names only by per-source opt-in + (`custom.ban_authors: true`, for a collaboration that is itself secret; + the common confidential types — own submitted work, purchased reports, + private repos — are protected by title/aliases, and banning their authors + would mostly ban legitimate names) — and maintains a fenced generated + block in the repo's untracked `.abcd/.work.local/private-names.txt`; + hand-added lines survive. 3. The committed `.githooks/pre-commit` guard (itd-74's private layer) **auto-refreshes** the generated block when the corpus is present (no-op otherwise — CI, fresh clones), then blocks any staged line matching — the diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index d61102a9..75c8bef5 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -40,3 +40,8 @@ parallel-agent merge contention bites. - 2026-07-08 — `~/.abcd/` blessed as abcd's user-level home (fourth tier, additive to repo `.abcd/`), path configurable; relocation wizard recorded as itd-77. +- 2026-07-08 — Author bans FLIPPED to opt-in (`ban_authors: true`), superseding + today's default-on decision: the actual corpus population (own submitted + work, purchased reports, private repos) makes author bans near-pure false + positives — they would ban the user's own name — while title/alias patterns + carry the real protection. From 72026e70eb5328d744ace1c8acf6332c9105b56e Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Wed, 8 Jul 2026 13:53:48 +0100 Subject: [PATCH 04/11] docs: add itd-78 (intent dependency graph); pilot severity + edges on itd-76/77 Severity says how much a thing matters; the graph says when to build it. Intents declare severity (capture-ledger enum) and blocked_by / builds_on edges; effective priority is derived by priority inheritance (a minor blocker of a major intent computes to major) and never stored, per the directory-is-state doctrine. Phases keep sequencing authority (adr-9); lint gains graph checks (cycles, dangling ids, phase-order violations, ignored inversions). Records an earlier-session decision that severity must not determine priority. Assisted-by: Claude:claude-fable-5 --- .../drafts/itd-76-source-provenance-ledger.md | 3 ++ .../drafts/itd-77-relocatable-user-home.md | 3 ++ .../drafts/itd-78-intent-dependency-graph.md | 36 +++++++++++++++++++ .abcd/work/DECISIONS.md | 7 ++++ 4 files changed, 49 insertions(+) create mode 100644 .abcd/development/intents/drafts/itd-78-intent-dependency-graph.md diff --git a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md index ac7834aa..8e5cdc99 100644 --- a/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md +++ b/.abcd/development/intents/drafts/itd-76-source-provenance-ledger.md @@ -5,6 +5,9 @@ spec_id: null kind: standalone suggested_kind: null reclassification_history: [] +severity: major +blocked_by: [itd-74] +builds_on: [itd-77] --- # abcd Lets You Consult Sources You Cannot Cite — and Remembers Every Debt diff --git a/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md b/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md index eeca4c25..cfce2fe4 100644 --- a/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md +++ b/.abcd/development/intents/drafts/itd-77-relocatable-user-home.md @@ -5,6 +5,9 @@ spec_id: null kind: standalone suggested_kind: null reclassification_history: [] +severity: minor +blocked_by: [] +builds_on: [] --- # abcd's User-Level Home Lives Where You Want It diff --git a/.abcd/development/intents/drafts/itd-78-intent-dependency-graph.md b/.abcd/development/intents/drafts/itd-78-intent-dependency-graph.md new file mode 100644 index 00000000..c6b8be2c --- /dev/null +++ b/.abcd/development/intents/drafts/itd-78-intent-dependency-graph.md @@ -0,0 +1,36 @@ +--- +id: itd-78 +slug: intent-dependency-graph +spec_id: null +kind: standalone +suggested_kind: null +reclassification_history: [] +severity: minor +blocked_by: [] +builds_on: [] +--- + +# abcd Knows What to Build First — Even When It's Something Small + +## Press Release + +> **Severity says how much a thing matters; the dependency graph says when to build it — abcd keeps the two honest and computes the order.** A roadmap stalls quietly when a small piece of plumbing blocks a big feature and nobody notices, because everything is ranked by how important it *looks*. abcd separates the two axes. Each intent **declares** two kinds of fact known at capture time: its **severity** (`nitpick | minor | major | critical` — the same enum the issue ledger already uses) and its **edges** — `blocked_by` (cannot ship before) and `builds_on` (cheaper or better if the other exists first). From those declarations abcd **derives** what it never stores: effective priority, by *priority inheritance* — an intent's effective priority is the maximum of its own severity and the severity of everything it transitively blocks. A minor intent holding up a major one jumps the queue while staying honestly minor; the classic priority inversion is resolved the way schedulers have always resolved it. The graph is linted, not trusted: cycles, dangling ids, a phase that scopes an intent before its blocker's phase, and the silent stall — a major intent scheduled while its minor blocker sits unscheduled — all fail deterministically. +> +> "I kept re-discovering, one grilling at a time, that the unglamorous banlist work was the thing to do first," said Alice, a maintainer. "Now the graph tells me: it's minor, and it computes to major, because everything I actually care about sits behind it." Her collaborator Bob stopped arguing rank entirely: "We only debate two things now — how severe, and what blocks what. The order falls out." + +## Why This Matters + +Severity and priority get conflated because most trackers offer one field for both, and the conflation has a failure mode in each direction: rank by severity and small blockers starve everything behind them; hand-rank priority and the number silently goes stale as the graph changes. The resolution is the same one this record already applies elsewhere — declare inputs, derive outputs, lint the consistency. Declared severity and declared edges are capture-time human judgments (legitimate frontmatter, like `kind`); computed priority is a cached mirror if written down, so it never is — the same doctrine that removed `status:` fields (directory is state) keeps effective priority out of frontmatter. Sequencing authority does not move: phase docs' `## Scope` sections remain the single source of truth for what ships when (adr-9); the graph's job is to make a *contradictory* schedule un-lintable, not to generate the schedule. + +## What It Looks Like + +- **Two declared fields, one shared enum.** Intent frontmatter gains `severity:` (the capture ledger's `nitpick|minor|major|critical`) and the edge lists `blocked_by:` / `builds_on:` (itd-N references). Deliberate non-dependencies stay in prose — the schema records edges, the press release records why an edge is absent (as itd-76 does for itd-16). +- **`abcd intent graph`** computes and reports, never persists: effective priority via inheritance (max of own severity and the severity of all transitively blocked intents), tie-broken by transitive unblock count, then capture order. Output is a ranked build-next list with the inheritance chain shown ("itd-74: minor, effective major ← blocks itd-76"). +- **Lint, deterministic.** Record lint gains graph checks: unknown itd references, cycles, an intent phase-scoped earlier than its blocker's phase, and priority inversion the schedule ignores (a scoped intent whose transitive blocker is unscheduled). All computable from frontmatter plus the phase docs' `## Scope` sections. +- **Dogfooding starts now.** itd-76 (`severity: major`, `blocked_by: [itd-74]`, `builds_on: [itd-77]`) and itd-77 (`severity: minor`, no edges) carry the pilot frontmatter; the derived result matches what two grillings concluded by hand — itd-74 first, itd-77 alongside. + +## Open Questions + +- Whether severity is required on every intent at capture or optional-with-default (`minor`), and whether existing intents get a one-shot severity sweep or accrue it opportunistically. +- Whether `builds_on` (soft edges) participates in priority inheritance at reduced weight or only in reporting — hard edges only is the conservative start. +- Whether the intent surface contract binds these fields at `/abcd:intent plan` time (like `kind`) or leaves them mutable for the intent's life — edges appear as understanding grows, which argues mutable. diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index 75c8bef5..32745920 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -45,3 +45,10 @@ parallel-agent merge contention bites. work, purchased reports, private repos) makes author bans near-pure false positives — they would ban the user's own name — while title/alias patterns carry the real protection. +- 2026-07-08 — Severity ≠ priority (records an earlier-session decision): + intents declare `severity` (capture-ledger enum) and edges (`blocked_by`, + `builds_on`); effective priority is DERIVED via priority inheritance (max of + own severity and severity of everything transitively blocked) and never + stored — a minor blocker of a major intent jumps the queue while staying + minor. Phases keep sequencing authority (adr-9); lint makes contradictory + schedules fail. Recorded as itd-78; piloted on itd-76/77. From e04202ed6c0b945c57ea3a11d3ff31aebb657206 Mon Sep 17 00:00:00 2001 From: REPPL <77722411+REPPL@users.noreply.github.com> Date: Wed, 8 Jul 2026 14:21:21 +0100 Subject: [PATCH 05/11] docs: land high-confidence dependency edges from the 64-intent sweep (itd-78 pilot) Seven parallel assessments proposed severity + edges for every draft, planned, and discipline intent. This commit lands only the edges whose evidence is an explicit statement in the intent's own prose; severities and low-confidence edges await human ratification. Edges into superseded targets (itd-47) and direction-suspect edges were withheld. Assisted-by: Claude:claude-fable-5 --- .../intents/disciplines/itd-37-modification-grammar.md | 2 ++ .../intents/disciplines/itd-5-prompt-quality-additions.md | 1 + .abcd/development/intents/drafts/itd-10-purge-uninstall.md | 1 + .../intents/drafts/itd-14-prompt-registry-versioning.md | 2 ++ .../intents/drafts/itd-16-hash-chain-merkle-audit.md | 1 + .../intents/drafts/itd-17-model-effectiveness-tracking.md | 1 + .../intents/drafts/itd-21-no-lifeboat-scaffolding.md | 1 + .abcd/development/intents/drafts/itd-22-opencode-portability.md | 1 + .../intents/drafts/itd-25-dredge-cross-corpus-synthesist.md | 2 ++ .abcd/development/intents/drafts/itd-26-loot-oss-vendor.md | 1 + .../intents/drafts/itd-30-design-fictions-as-intent-format.md | 2 ++ .../intents/drafts/itd-33-agent-communication-infrastructure.md | 2 ++ .../intents/drafts/itd-35-lifeboat-integrity-audit.md | 2 ++ .../intents/drafts/itd-39-scope-aware-memory-retrieval.md | 1 + .../intents/drafts/itd-44-fourth-intent-kind-decision.md | 1 + .../intents/drafts/itd-61-brief-change-derivation.md | 1 + .../development/intents/drafts/itd-62-pluggable-safety-gate.md | 1 + .../intents/planned/itd-27-grill-skill-and-glossary.md | 1 + .../development/intents/planned/itd-28-rp-reviews-into-flow.md | 1 + .../intents/planned/itd-29-autonomous-run-resilience.md | 2 ++ .abcd/development/intents/planned/itd-34-three-intent-kinds.md | 1 + .../development/intents/planned/itd-42-coherence-aware-grill.md | 2 ++ .../planned/itd-46-abcd-intent-quoted-text-create-symmetric.md | 1 + .../planned/itd-48-intent-fidelity-reviewer-roles-2-3.md | 1 + .../intents/planned/itd-50-loop-toward-acceptance.md | 2 ++ .../intents/planned/itd-6-rp-mcp-only-integration.md | 1 + .../intents/planned/itd-63-setup-wizard-explains-installs.md | 1 + .../intents/planned/itd-65-launch-preflight-gate-suite.md | 2 ++ .../intents/planned/itd-67-installable-versioned-plugin.md | 1 + .../intents/planned/itd-72-launch-ship-tier-b-publishing.md | 1 + 30 files changed, 40 insertions(+) diff --git a/.abcd/development/intents/disciplines/itd-37-modification-grammar.md b/.abcd/development/intents/disciplines/itd-37-modification-grammar.md index 6036f73a..0f8e77ae 100644 --- a/.abcd/development/intents/disciplines/itd-37-modification-grammar.md +++ b/.abcd/development/intents/disciplines/itd-37-modification-grammar.md @@ -6,6 +6,8 @@ kind_notes: "Cross-cutting modification-grammar gate; applied at every spec plan suggested_kind: null spec_id: null reclassification_history: [] +blocked_by: [itd-36] +builds_on: [itd-1] --- # Every Spec Externalises How It Should And Shouldn't Be Modified diff --git a/.abcd/development/intents/disciplines/itd-5-prompt-quality-additions.md b/.abcd/development/intents/disciplines/itd-5-prompt-quality-additions.md index 1575225d..7447aa3f 100644 --- a/.abcd/development/intents/disciplines/itd-5-prompt-quality-additions.md +++ b/.abcd/development/intents/disciplines/itd-5-prompt-quality-additions.md @@ -7,6 +7,7 @@ suggested_kind: null spec_id: null reclassification_history: - { date: 2026-05-07, from: standalone, to: discipline, reason: "Reclassified per the three-intent-kinds change: itd-5 is a rule that applies to every agent spec — `prompt_version` frontmatter, one-shot pre-flight at lock-time, injection canaries — not a feature with a user moment of its own. Discipline shape (## Rule + ## Why) fits." } +blocked_by: [itd-1] --- # Every Agent Prompt Carries Its Version, Earns Its Lock, And Survives Injection diff --git a/.abcd/development/intents/drafts/itd-10-purge-uninstall.md b/.abcd/development/intents/drafts/itd-10-purge-uninstall.md index 86a1ea4f..7b2f8e09 100644 --- a/.abcd/development/intents/drafts/itd-10-purge-uninstall.md +++ b/.abcd/development/intents/drafts/itd-10-purge-uninstall.md @@ -5,6 +5,7 @@ spec_id: null kind: standalone suggested_kind: null reclassification_history: [] +builds_on: [itd-13] --- # Full Removal When You Mean It diff --git a/.abcd/development/intents/drafts/itd-14-prompt-registry-versioning.md b/.abcd/development/intents/drafts/itd-14-prompt-registry-versioning.md index 3e4ee32b..01fff95f 100644 --- a/.abcd/development/intents/drafts/itd-14-prompt-registry-versioning.md +++ b/.abcd/development/intents/drafts/itd-14-prompt-registry-versioning.md @@ -5,6 +5,8 @@ spec_id: null kind: standalone suggested_kind: null reclassification_history: [] +blocked_by: [itd-5] +builds_on: [itd-15] --- # Prompts Are Versioned Like Code diff --git a/.abcd/development/intents/drafts/itd-16-hash-chain-merkle-audit.md b/.abcd/development/intents/drafts/itd-16-hash-chain-merkle-audit.md index b63fc8f9..e96398df 100644 --- a/.abcd/development/intents/drafts/itd-16-hash-chain-merkle-audit.md +++ b/.abcd/development/intents/drafts/itd-16-hash-chain-merkle-audit.md @@ -5,6 +5,7 @@ spec_id: null kind: standalone suggested_kind: null reclassification_history: [] +builds_on: [itd-9] ---