Context document: index view, narrowed ?model=/?terms=, census behind include=check

CHANGELOG.md

@@ -6,6 +6,42 @@ loosely follows [Keep a Changelog](https://keepachangelog.com/); dates are
 
 ## [Unreleased]
 
+### Added: `?view=index` — a whole-cell projection cheap enough for an agent to read (ADR 0012 §4, ADR 0017 §2, ADR 0018 §7 amendments)
+
+On a real 42-export/47-dataset cell, `GET /context` measured 429 KB
+compact — `exports[]` alone was 383 KB (`semantic` 158 KB, `schema` 122 KB,
+`check` 87 KB) — and agents refused to read it; `?model=`/`?terms=` fared no
+better (467 KB and 431 KB respectively, of which the actually-requested
+content was 38 KB and 1.4 KB). Additive; `datamk_context` stays 4.
+
+- **`?view=index` / `datamk context --view index`.** Projects every
+  `exports[]` entry to identity, claims (`description`, `grain`,
+  `freshness`, `from`), affordances (`query`/`binding`, `deployed`),
+  declared column *names* (`columns[]`, in place of `schema`'s full column
+  bodies), and bound Ossie dataset names (`semantic_datasets[]`, in place
+  of `semantic[]`'s full bodies) — everything cell-level stays. Composes
+  with `include`/`terms`/`model`; refused (400) on `/context/<route>`, a
+  single-export door already being the index's whole point. New
+  `index_request` affordance, always present. New `~index` `ETag` suffix.
+  MCP resource `datamk://<mount>/context/index`.
+- **`?model=` and `?terms=` (without a route) now apply the same
+  projection automatically** — this door never narrows by route, so asking
+  for one model or a term subset has no reason to drag every export's full
+  body along. `/context/<route>?terms=` is exempt: that door's whole point
+  is the route's full export. `datamk context --model`/`--terms` (without
+  `--export`) match.
+- **The per-column census (`exports[].check.columns`) moves behind
+  `?include=check`** — omitted by default (including under `?view=index`),
+  present under `?include=check`, and always inlined in the portable
+  `datamk context` emission (no `--no-check` flag; a file cannot be
+  re-requested). The rollup (`check`, `grain`, `rows`, `distinct_grain`,
+  `null_rows`, `at`) stays on every record. `include` becomes `docs` |
+  `check`; `included` lists `check` when inlined. New `~check` `ETag`
+  suffix.
+- Measured on a synthetic 40-export/30-column/one-dataset-per-export cell:
+  `?view=index` and `?model=` both land under 10% of the full document's
+  bytes.
+
 ### Added: `semantic_model:` — Apache Ossie ingest (ADR 0018)
 
 datamake ingests business meaning it does not author: `semantic_model:`

docs/adr/0012-cell-context-document.md

@@ -841,3 +841,54 @@ reused, not reimplemented. What would reverse it: a design partner needing
 MCP against a hosted cell they cannot run locally (then `POST /mcp` on
 `serve`, same module, inheriting bearer auth); agents demonstrably unable
 to compose `describe_export` → `query_export` in two hops.
+
+## Amendment (2026-09-11): `?view=index` — the index projection (context bloat)
+
+Measured on a real 42-export/47-dataset cell: default `/context` is 429 KB
+compact, of which `exports[]` is 383 KB (`semantic` 158 KB, `schema` 122 KB,
+`check` 87 KB, `description` 25 KB, `deployed` 13 KB) — agents refuse it.
+§2's "the document is flat, one schema" stands; what was missing was a
+*view* of that one schema cheap enough to pick a route from before paying
+for its body.
+
+- **`?view=index` / `datamk context --view index`** — closed vocabulary
+  `full` (default) | `index`. `index` returns the full document with every
+  `exports[]` entry projected through `ExportDoc::to_index`: identity
+  (`name`, `version`, `route`, `contract`), the claims that fit in a
+  sentence (`description`, `grain`, `freshness`, `from`), the affordances
+  (`query`/`binding`, `deployed`), declared column *names* (`columns[]`,
+  §3's `schema` minus type/unit/description/`from`), and bound Ossie
+  dataset names (`semantic_datasets[]`, `model/dataset`, §7's `semantic[]`
+  minus every field/relationship/metric body). Every cell-level field
+  (`semantic_models[]`, `definitions[]`, `upstreams[]`, `docs[]` identity,
+  `build`, `source_check` rollup, `data`, `notes`) stays — this is an
+  `exports[]`-only projection, not a second document.
+- **Composes**, never collapses: `include`, `terms`, `model` all still work
+  under `view=index` — the projection runs on whatever `exports[]` those
+  narrowed it to. Refused on `/context/<route>` (400): a single-export door
+  is already the index's whole point, so the combination names something
+  incoherent, the same refusal `?model=` already draws there.
+- **`index_request`** joins `include_request`/`definitions_request` as a
+  constant, always-present affordance, relative per §2's RFC 3986 rule.
+- **ETag** gains a `~index` component, same style as `~docs`/`~model`; the
+  plain default response is unaffected byte for byte.
+- **One schema, still.** `ExportDoc` gained two fields (`columns`,
+  `semantic_datasets`), both empty and omitted on the full document — no
+  second struct, no second document to keep in sync. A consumer that has
+  only ever parsed `view=index` reads `view=full` unmodified; it just sees
+  more fields populated.
+- **MCP:** `datamk://<mount>/context/index`, alongside `context` and
+  `context/<route>` — the same projection over stdio.
+
+Measured on a synthetic 40-export/30-column/one-dataset-per-export cell:
+`?view=index` and `?model=` both land under 10% of the full document's
+bytes (`serve::smoke::
+view_index_and_model_are_under_a_tenth_of_the_full_document_on_a_synthetic_cell`).
+See the ADR 0017 §2 and ADR 0018 §7 amendments (same date) for `?model=`
+and `?terms=` (without a route) applying this same projection
+automatically — the other half of the same measurement.
+
+Premises: an agent's first move is almost always "what exists," not "give
+me everything" — the index answers the first cheaply. What would reverse
+it: a client that always wants the full body anyway, for which `view=index`
+is simply unused, at zero cost to it.

docs/adr/0017-definitions.md

@@ -342,3 +342,33 @@ exits 0: the emission is truthful about its own state. Falsified if
 operators keep misreading a stale emission as validated — that argues for a
 non-zero exit, which is a behavior change this amendment deliberately does
 not take.
+
+## Amendment (2026-09-11): `?model=`/`?terms=` (no route) carry the index projection
+
+§2's "the document keeps its shape" still holds — one schema, additive
+fields, `datamk_context` stays 4 — but on the un-routed `/context` door,
+asking for `terms=` alone was paying for every export's full `schema`,
+`check`, `probe`, and `semantic[]` body to answer a lookup that only ever
+touches `definitions[]`/`docs[]`. Measured on the same 42-export/47-dataset
+cell ADR 0012's `view=index` amendment measures: `?terms=` is 431 KB, of
+which the matches (`definitions[]` + `semantic_matches[]`) are 1.4 KB.
+
+`GET /context?terms=...` and `GET /context?model=...`, absent a route, now
+also project `exports[]` through `ExportDoc::to_index` (ADR 0012 §4
+amendment 2026-09-11) — automatically, without `view=index` — because this
+door never narrows by route, so the narrower ask (a term subset, one model)
+has no reason to drag the wider one along. `datamk context --model`/
+`--terms` (without `--export`) match. **`/context/<route>?terms=`
+is exempt**: §3's composition rule already holds one route's document to
+its full export, and that full export is the whole point of asking a route
+door — projecting it there would take away the one thing a route-scoped
+request is for.
+
+The document's shape is unchanged by this — same `ExportDoc` schema, same
+fields, just fewer of them populated for this request. There is no escape
+hatch back to full `exports[]` under `terms=`/`model=`: the narrowed
+request is the signal that `exports[]` is not what's being asked for.
+Falsified if a caller demonstrably wants `terms=`/`model=` composed with
+full export bodies — the fix then is a route-scoped request
+(`/context/<route>?terms=`, exempt above) or the plain default document,
+not a third parameter on this door.

docs/adr/0018-ossie-ingest.md

@@ -262,3 +262,29 @@ not-found message.
    answer is `unverified`, never a join planner.
 3. Ossie 0.2 moves metrics to dataset-level measures ⇒ the version pin
    fails loudly on upgrade and the lookup keys change shape under authors.
+
+## Amendment (2026-09-11): `?model=`/`?terms=` (no route) project `exports[]` too
+
+§7's table says `/context?model=<name>` returns "one semantic model in
+full, composable with `terms`" — true, but on the real 42-export/47-dataset
+design-partner cell it arrived beside all 42 exports' full `schema`/
+`check`/`semantic[]` bodies anyway: 467 KB, of which the requested model
+was 38 KB. The door answered "the one thing you asked for" and "everything
+else" in the same breath.
+
+`GET /context?model=<name>` and `GET /context?terms=...`, absent a route,
+now project `exports[]` through `ExportDoc::to_index` (ADR 0012 §4
+amendment 2026-09-11, same date) — every export reduced to identity, its
+claims, `columns[]` (names), and `semantic_datasets[]` (`model/dataset`
+names) in place of `semantic[]`'s full dataset/field/relationship/metric
+bodies. `semantic_model`/`semantic_matches` themselves are unaffected —
+`?model=` still returns that one model in full; the projection only trims
+the tag-along `exports[]`. `datamk context --model`/`--terms` (without
+`--export`) match. `/context/<route>?terms=` keeps the route's full export
+— see the ADR 0017 §2 amendment (same date) for the full reasoning, which
+applies identically here since both doors share `?terms=`'s grammar.
+
+MCP is unaffected structurally: `describe_export` already narrows to one
+route (never carries the whole-cell `exports[]` to begin with), and the new
+`datamk://<mount>/context/index` resource (ADR 0012 §4 amendment) is the
+whole-cell projection's own door.