Skip to content
Merged
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,7 @@ kind_notes: "Cross-cutting modification-grammar gate; applied at every spec plan
suggested_kind: null
spec_id: null
reclassification_history: []
blocked_by: [itd-36]
builds_on: [itd-1]
builds_on: [itd-1, itd-36]
severity: major
---

Expand Down Expand Up @@ -49,7 +48,7 @@ The semantic enforcement is genuine LLM-judgement work. The discipline owns the

**Why all specs, no "trivial" carve-out.** "Trivial" cannot be cleanly specified without inviting loophole-driven exemptions. A `## Modification Grammar` section that says *"this spec adds a config flag with no extension surface; modifications are limited to renaming the flag; the rule is: never bind config-flag names to public API"* is **valuable** — documenting absence-of-extension-points IS modification grammar. Required for all; trivial specs produce short Modification Grammar sections, not absent ones.

**Why memory routing as secondary index, not primary store.** The dominant access pattern is "I'm modifying spec spc-N, give me spc-N's modification grammar" — that's a lookup the spec file already serves trivially. Memory routing earns its keep on the long-tail: *"how has our modification grammar of dispatch evolved across spc-1, spc-3, spc-7?"* Cheap to add (`principle-distiller` already exists post-itd-36); worth it; not load-bearing for the common case. In-spec capture is the primary store.
**Why memory routing as secondary index, not primary store.** The dominant access pattern is "I'm modifying spec spc-N, give me spc-N's modification grammar" — that's a lookup the spec file already serves trivially. Memory routing earns its keep on the long-tail: *"how has our modification grammar of dispatch evolved across spc-1, spc-3, spc-7?"* Cheap to add (`principle-distiller` already exists post-itd-36); worth it; not load-bearing for the common case. In-spec capture is the primary store. This is also why itd-36 is a `builds_on` edge, not a blocker: the capture + enforcement half (`MG001`-`MG004`, the Phase 0 discipline registration) ships independently of itd-36, and only the extraction-to-memory trigger waits on its substrate — the partial-ship fallback below is the designed behaviour, not a contingency.

## What's In Scope

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ abcd doesn't need to host a separate audit service: the chain lives in `.abcd/lo
- **`audit.schema.json`** — the public format for chain segments and the verification report.
- **Disembark integration**: `/abcd:disembark` includes the latest chain segments in the lifeboat under `_provenance.json` `audit_chain_root`. The chain root hash anchors the lifeboat's authenticity.
- **Embark integration**: `/abcd:embark` verifies the lifeboat's chain root before unpacking; mismatch aborts the unpack with the offending segment named.
- **Launch integration** (per itd-launch): launch payload manifest excludes the chain by default (sensitive); explicit opt-in via `.abcd/launch.allow` if a release should ship its own chain.
- **Launch integration** (per itd-66): launch payload manifest excludes the chain by default (sensitive); explicit opt-in via `.abcd/launch.allow` if a release should ship its own chain.
- **Opt-in via `.abcd/config.json`** — `oracle.audit.chain.enabled = true | false` (default `false` to avoid surprising users; flip to `true` for compliance-sensitive projects).

## What's Out of Scope
Expand Down
16 changes: 8 additions & 8 deletions .abcd/development/intents/planned/itd-36-memory-unification.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ severity: major

# Knowledge That Compounds, Not Knowledge That Re-Derives

> **Superseded in part by [adr-28](../../decisions/adrs/0028-single-repo-curated-release.md) (packaging framing; supersedes adr-18).** This intent describes the restrictive-licence gate as a `/abcd:launch` payload gate (the "launch-gate refuses to publish under `.abcd/memory/sources/`" phrasing throughout, incl. the GWT criteria below). adr-28 packages the curated release to exclude `.abcd/**` **wholesale**, so launch is **not** the gate's consumer — the gate's real consumer is the lifeboat (`/abcd:disembark`), future/inert at launch. Read every "launch-gate" / "`/abcd:launch` refuses" reference here as the **lifeboat** licence gate; the canonical current-state framing lives in the brief (`05-internals/09-provenance-substrate.md § 4`, `07-memory.md § 4`, `04-surfaces/04-launch.md § 2`).
> **Packaging framing per [adr-28](../../decisions/adrs/0028-single-repo-curated-release.md) (supersedes adr-18).** The restrictive-licence gate's consumer is the **lifeboat** (`/abcd:disembark`), future/inert at launch: the curated release excludes `.abcd/**` wholesale, so `/abcd:launch` never evaluates what the gate guards. The canonical framing lives in the brief (`05-internals/09-provenance-substrate.md § 4`, `07-memory.md § 4`, `04-surfaces/04-launch.md § 2`).

## Press Release

Expand All @@ -27,20 +27,20 @@ There's a real gap: **per-project durable knowledge has multiple legitimate upst

**The reframe lands cleanly because abcd doesn't need a new namespace.** `.abcd/memory/` already exists as the per-project compounding-curated artefact. Widening its upstream funnel from "vendor session memory only" to "session memory + external sources + reviews + notes + dredge synthesis + spec-extracted theory" is structurally smaller than creating a new top-level namespace — and avoids the visibility-rule violations that would come with a separate `.abcd/knowledge/` directory (per the locked decision in [`05-internals/03-configuration.md § 1`](../../brief/05-internals/03-configuration.md): "no exceptions to the visibility rule").

**Default-no-originals is what makes ingest copyright-safe.** External sources may be paywalled, NDA'd, or under restrictive licences. Storing the original creates a copyright-laundering vector. Default-no-originals + bounded per-page quotation (≤5%, no contiguous span >150 words) + cumulative source-coverage lint (≤25% deduplicated coverage across all pages citing the same source, span-level dedup) closes the laundering vector for the common case. `--keep-original` is the explicit opt-in for sources the user owns or has clear redistribution rights for; the launch-gate then refuses to publish anything under `.abcd/memory/sources/` unless an explicit allowlist entry exists.
**Default-no-originals is what makes ingest copyright-safe.** External sources may be paywalled, NDA'd, or under restrictive licences. Storing the original creates a copyright-laundering vector. Default-no-originals + bounded per-page quotation (≤5%, no contiguous span >150 words) + cumulative source-coverage lint (≤25% deduplicated coverage across all pages citing the same source, span-level dedup) closes the laundering vector for the common case. `--keep-original` is the explicit opt-in for sources the user owns or has clear redistribution rights for; the restrictive-licence gate (consumed by the lifeboat `/abcd:disembark`, per adr-28) then refuses to surface anything under `.abcd/memory/sources/` unless an explicit allowlist entry exists.

**Loot substrate reuse compresses cost.** itd-26 `/abcd:loot` (per itd-26) and `/abcd:memory ingest` need the same machinery: licence detection, citation generation, source-hash registry, launch-gate licence gates. This intent ships the substrate as a separable component spec at [`05-internals/09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md). itd-26's verb sits on top — no re-implementation, no fragile bespoke licence layer.
**Loot substrate reuse compresses cost.** itd-26 `/abcd:loot` (per itd-26) and `/abcd:memory ingest` need the same machinery: licence detection, citation generation, source-hash registry, the lifeboat's restrictive-licence gates. This intent ships the substrate as a separable component spec at [`05-internals/09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md). itd-26's verb sits on top — no re-implementation, no fragile bespoke licence layer.

## What's In Scope

- **`/abcd:memory` command** with sub-verbs `ingest` / `ask` / `lint`. Bare invocation renders state per the universal bare-command-as-render discipline. Surface contract: [`04-surfaces/07-memory.md`](../../brief/04-surfaces/07-memory.md). Substrate spec: [`05-internals/07-memory.md`](../../brief/05-internals/07-memory.md).
- **Schema extension on `.abcd/memory/`** (NOT migration): three new conventional sibling files (`index.md` generated catalog, `log.md` append-only ingest history, `contradictions.md` curator-surfaced register) plus typed `source:` frontmatter on every memory page. Existing flat-named pages preserved; schema extends in place.
- **Typed `source:` frontmatter** with closed-enum source-class taxonomy: `session_memory` / `external_pdf` / `external_transcript` / `external_article` / `oracle_review` / `work_notes` / `issue_ledger` / `dredge_synthesis` / `spec_modification_grammar` / `modification_grammar`. Every external source carries `citation` + `licence` + `source_hash` + `ingested_at`.
- **Default-no-originals + `--keep-original`**: ingest reads → distils → discards by default; flag opts into storage at `.abcd/memory/sources/<sha256>.<ext>`; launch-gate refuses publish under `sources/` without explicit allowlist.
- **Default-no-originals + `--keep-original`**: ingest reads → distils → discards by default; flag opts into storage at `.abcd/memory/sources/<sha256>.<ext>`; the lifeboat licence gate refuses to surface `sources/` files without an explicit allowlist entry.
- **Quotation budgets, two layers**: per-page (`MQ001`: ≤5% per page, no >150-word contiguous span) + cumulative source-coverage (`MQ002`: ≤25% deduplicated coverage across all pages citing one `source_hash`, span-level dedup, configurable threshold).
- **Source-class lints**: `MS001` (advisory: single-source-class synthesis is low cross-validation), `MS002` (blocks: cross-class synthesis without `weighting_note` field), `ML001` (blocks: `external_*` page missing `licence` field; `unknown` is acceptable but must be explicit).
- **Curator role on `principle-distiller`** (existing agent; itd-31 precedent — agent count stays 15). Inputs extended: external-source ingest pipeline added alongside existing session-memory + reviews + work-dir + code-rescuer inputs. New output mandate: typed `source:` frontmatter on every emitted page. New lint pass: `MQ`/`MS`/`ML` codes.
- **Provenance/licence substrate** at [`05-internals/09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md) — separable spec; SPDX licence detection; citation generation; source-hash registry at `.abcd/memory/.sources_index.json` (regenerable per the lifecycle taxonomy); launch-gate licence gates. Shared with itd-26 loot.
- **Provenance/licence substrate** at [`05-internals/09-provenance-substrate.md`](../../brief/05-internals/09-provenance-substrate.md) — separable spec; SPDX licence detection; citation generation; source-hash registry at `.abcd/memory/.sources_index.json` (regenerable per the lifecycle taxonomy); the lifeboat's restrictive-licence gates. Shared with itd-26 loot.
- **Karpathy citation** in [`05-internals/07-memory.md`](../../brief/05-internals/07-memory.md) (component description) + [`research/related-work.md`](../../research/related-work.md#karpathy-llm-wiki--pattern-source-for-abcdmemory) (prior-art table). NOT in `01-product/03-mental-model.md` (per idea-1 R5 review: mental model is for axes, not components; six-week-old citations don't belong in the project's most-stable file).
- **Brief edits**: artefact-lifecycle taxonomy added to [`05-internals/04-universal-patterns.md § 8`](../../brief/05-internals/04-universal-patterns.md#8-artefact-lifecycle-taxonomy) (regenerable / append-only / compounding-curated). Reserved-vocabulary entry in [`02-constraints/04-naming.md`](../../brief/02-constraints/04-naming.md) for `source.class` + lifecycle classes.

Expand All @@ -60,7 +60,7 @@ There's a real gap: **per-project durable knowledge has multiple legitimate upst

- **Given** an external source with declared SPDX licence in a `LICENSE` file, **when** the user runs `/abcd:memory ingest <path>`, **then** the resulting memory page(s) carry `source.licence: <spdx-id>` AND the original is NOT stored at `.abcd/memory/sources/` AND a registry entry exists at `.abcd/memory/.sources_index.json[<sha256>]` whose `consumers` map contains the `memory` key.
- **Given** an external source ingested without `--keep-original`, **when** the user later runs `/abcd:memory ask <question>` referencing that source, **then** the answer cites the source by `(class, citation, source_hash)` but never reproduces the original; if the original is needed for re-distillation, the user is prompted to re-ingest.
- **Given** an external source ingested with `--keep-original`, **when** `/abcd:launch` runs, **then** launch refuses to publish unless `.abcd/memory/sources/<sha256>.<ext>` is on the launch-payload allowlist (`.abcd/launch-allowlist.json`).
- **Given** an external source ingested with `--keep-original`, **when** `/abcd:disembark` runs the restrictive-licence gate, **then** the gate refuses to surface `.abcd/memory/sources/<sha256>.<ext>` unless `.abcd/launch-allowlist.json` explicitly names it (the allowlist re-includes files into the gate's own evaluation input, never into the launch payload — adr-28).
- **Given** a memory page with `source.class: external_pdf` and ≥150-word contiguous quotation, **when** `/abcd:memory lint` runs, **then** the lint emits `MQ001` and points at the offending span.
- **Given** N memory pages collectively citing one `source_hash` with cumulative deduplicated coverage ≥25%, **when** `/abcd:memory lint` runs, **then** the lint emits `MQ002` and refuses any further quotation block from that source until coverage drops below the threshold (via summarisation rewrite of existing pages).
- **Given** a memory page with `source.class: session_memory` only (no cross-class inputs), **when** `/abcd:memory lint` runs, **then** the lint emits `MS001` (low cross-validation; advisory only — does not block).
Expand All @@ -69,7 +69,7 @@ There's a real gap: **per-project durable knowledge has multiple legitimate upst
- **Given** an existing pre-itd-36 project with flat-named memory files, **when** `/abcd:ahoy` runs after itd-36 ships, **then** the existing files are NOT renamed; `index.md` is generated over them; `source.class: session_memory` is backfilled as the default.
- **Given** dredge synthesis output (itd-25), **when** dredge runs, **then** the dredge agent writes its synthesised entries to `.abcd/memory/<type>_<domain>_<slug>.md` with `source.class: dredge_synthesis` AND the per-run report stays at `.abcd/logbook/dredge/<ts>/`.
- **Given** the same source content is ingested by both `/abcd:memory ingest` (as documentation) and `/abcd:loot` (as code, per itd-26), **when** both have run, **then** the registry has one entry with `ingest_count: 2` whose `consumers` map holds both the `memory` and `loot` keys (memory and loot share one substrate registry).
- **Given** a publish payload includes a file with `citation.licence: GPL-3.0` AND the project is being published as MIT, **when** `/abcd:launch` runs, **then** launch refuses with a licence-mismatch error AND lists the offending file. Override via `--accept-licence-risk` logs the override in the launch report.
- **Given** the lifeboat's gated payload includes a file with `citation.licence: GPL-3.0` AND the project is being published as MIT, **when** `/abcd:disembark` runs the restrictive-licence gate, **then** the gate refuses with a licence-mismatch error AND lists the offending file. Override via `--accept-licence-risk` logs the override in the disembark report.
- **Given** this intent's cost-discipline boundary, **when** a promotion proposes adding `known_failure_modes` (or any other dynamic-state field) to memory frontmatter, **then** the proposal is rejected as scope creep — those fields belong to other intents (Frontier Awareness, per a separate intent, for `known_failure_modes`).

## Implementing specs
Expand All @@ -88,7 +88,7 @@ Per the idea-1 R5 review: the push replaces "≥3 projects in anger" (unachievab

1. **Research-shaped project** — heavy external PDFs, papers, talks. Test: does the synthesis stay coherent across 20+ external sources? Does the cumulative coverage lint trigger correctly? Are entity/topic pages distinguishable from a stack of PDF summaries?
2. **Tooling project** (e.g., abcd itself dogfooding) — almost no external sources; mostly session memory + reviews + notes. Test: does the schema feel cluttered with `source.class: session_memory` defaults? Does `index.md` provide value when most pages are principle-shaped?
3. **Mixed-licence project** — open-source docs + proprietary internal notes + paywalled academic sources. Test: do the licence gates and quotation budgets behave correctly? Does the launch-gate correctly refuse to publish proprietary content under `--keep-original`?
3. **Mixed-licence project** — open-source docs + proprietary internal notes + paywalled academic sources. Test: do the licence gates and quotation budgets behave correctly? Does the lifeboat licence gate correctly refuse to surface proprietary content under `--keep-original`?

If all three produce coherent `.abcd/memory/` outputs, this intent ships. If one produces sprawl or fails the licence gates, scope is reduced and the failed example becomes the primary debug target.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,13 @@ The 2026-06-02 grill (5 questions across Dialectic / Definition / Counterfactual
- **itd-44** (fourth intent kind / decision) and **itd-43** (terminology) — the product-thinker / facilitator two-vocabulary split this intent leans on.
- **`.work/issues.md` 2026-06-02** — the `[Design/Spec-candidate]` entry and the upstream `[Design/Decision-needed]` auto-drain recommendation that produced this intent; the auto-drainer is this intent's substrate.

## AC reconciliation (spc-52 — implementation complete)
## Prior Art

itd-50 was implemented by `spc-52-audit-loop-to-acceptance-modes` (tasks .1–.3). Each acceptance criterion above is **satisfied**; the open questions are **resolved** at plan + build time. No AC is deferred.
- `spc-52-audit-loop-to-acceptance-modes` — the predecessor implementation delivered this intent (tasks .1–.3); its AC reconciliation below is carried as design input per the brief's delivery-state provenance note. In this repo itd-50 is undelivered (nothing in the Go tree implements the audit loop) and ships only when this intent reaches `shipped/` with its own audit notes.

### Predecessor AC reconciliation (spc-52)

In the predecessor implementation each acceptance criterion above is satisfied and the open questions are resolved at its plan + build time; the table attributes which spc-52 task owns which behaviour.

| itd-50 Acceptance Criterion | Status | Where delivered |
|---|---|---|
Expand All @@ -115,4 +119,4 @@ itd-50 was implemented by `spc-52-audit-loop-to-acceptance-modes` (tasks .1–.3
| `UNACHIEVABLE` / rejection seeds `/abcd:intent grill` | Satisfied | spc-52.2 / .3 — both replan blocks name the grill seed |
| on-close hook stays a pure data function (no subprocess / oracle) | Satisfied | The mode logic rides the spc-43 drainer / policy layer; `intent_lifecycle` is untouched by the loop |

**Open questions (now resolved):** loop budget = one re-open+re-review cycle per iteration, default `3` (spc-52.1 § Decision context); replan surface = no `drafts/` move, a `why-unachievable` + replan block in `## Audit Notes` with the intent kept in `shipped/` (spc-52.2 R4); manual-verification sign-off = the receipt schema `{intent_id, machine_rollup, state, justification?, recorded_by_role, ts}` with the `rejected_wrong_criteria` state carrying the justification to the shared replan surface (spc-52.3 R5).
**Open questions (predecessor answers, to re-adjudicate at spec time):** loop budget = one re-open+re-review cycle per iteration, default `3` (spc-52.1 § Decision context); replan surface = no `drafts/` move, a `why-unachievable` + replan block in `## Audit Notes` with the intent kept in `shipped/` (spc-52.2 R4); manual-verification sign-off = the receipt schema `{intent_id, machine_rollup, state, justification?, recorded_by_role, ts}` with the `rejected_wrong_criteria` state carrying the justification to the shared replan surface (spc-52.3 R5).
Loading
Loading