From 5e9e55b489e3f5b43538b05b8787ccce45f49e1c Mon Sep 17 00:00:00 2001 From: "Claude (OGAR session)" Date: Fri, 5 Jun 2026 11:20:29 +0000 Subject: [PATCH] =?UTF-8?q?docs(adr):=20add=20ADR-024=20=E2=80=94=20Palett?= =?UTF-8?q?e256=20+=20HHTL=20codec=20as=20universal=20compression=20primit?= =?UTF-8?q?ive?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Names an existing primitive (three independent deployments + one empirical anchor) rather than proposing one. Companion to ADR-022 (The Firewall) and ADR-023 (IR-as-wire-truth). # The codec HHTL prefix (NiblePath / quadkey / class identity) ↓ establishes spatial / semantic frame within-frame values cluster ↓ quantize to 256-index palette ↓ const-table lookup (compile-time HHTL where possible) 1-byte index per element, sub-µs decode, zero heap allocation # Three already-deployed instances (verified, not narrative) - Security mesh — Binary16K = [u64; 256] in MedCare-rs/crates/medcare-analytics/src/graph_contract.rs:36; Hamming-popcount on _effectiveReaders at the inner / hot path; wired into production at column_mask_bridge.rs → medcare-server/state.rs:167, 265, 439. - Attention — bgz-tensor WeightPalette::build(…, 256) + AttentionTable::build (crates/bgz-tensor/examples/ compare_stacked_vs_i16.rs:90-92). - Distance — lance-graph-arm-discovery aerial codebook, measured ρ = 0.9973 vs cosine — the empirical anchor. The runtime side's BindSpace dissolution work (bardioc PR #18 / lance-graph PR #470) hinted at the same compression strategy via the Quintenzirkel qualia codebook (8 B → 1-2 B per row). # Adoption checklist for new domains 1. Identify the prefix (NiblePath / quadkey / class identity). 2. Identify the palette domain — which values cluster? 3. Build the palette + measure ρ-vs-reference. Target ≥ 0.99 to match the arm-discovery anchor. 4. Decode = const-table lookup (zero-allocation hot path). # The 256-ceiling escape hatches (in the ADR body, not in callouts) Per runtime-session's "name the escape hatch upfront" tightening so reviewers don't spend the first session asking about long-tail: - Per-tile / per-frame palettes — cheapest; different frame, different 256 entries. - Hierarchical palettes — coarser at higher quadkey levels, finer per leaf. Mirrors the tile pyramid + SH L0/L1 vs L2/L3 split in splat-fit. - Palette-64K upgrade — 2 bytes/index, for measured cases only (not speculation). # Two next-domain adopters queued - D-OSM-2 (OSM tag palette + tile-local coordinate quantization) — per lance-graph PR #473 (cesium-osm-substrate-v1.md). Reports ρ-vs-reference on first per-country PBF run per the runtime session's §11 follow-up commitment. - D-SPLAT-4 (SH-aware palette extension on the Gaussian3D carrier). # The falsifiable property "The same compile-time HHTL prefix + palette256 codec decodes (a) _effectiveReaders for row auth, (b) OSM way attributes at zoom-21 tile, and (c) FMA-bone SH coefficients at sub-microsecond per element with zero heap allocation." If that property holds across all three, the substrate is doing its job. If it fails on one, the substrate is leaking dialect into the codec. # Cross-references named in the ADR Three FINDING-grade anchors (per runtime-session's "name existing primitive rather than propose one" tightening): - lance-graph/.claude/board/EPIPHANIES.md:28 - lance-graph/.claude/knowledge/old-stack-capability-parity.md §3.39 - lance-graph-arm-discovery (ρ = 0.9973 vs cosine) Plus the file:line citations for all three production deployments + cross-references to ADR-022 / ADR-023 / THE-FIREWALL.md / HEALTHCARE-TRANSCODING.md / RDF-OWL-ALIGNMENT.md. PII abort-guard (word-boundary): CLEAN. https://claude.ai/code/session_01PBTGaPCSnnt6u3pjXpbLwY --- docs/ARCHITECTURAL-DECISIONS-2026-06-04.md | 180 +++++++++++++++++++++ 1 file changed, 180 insertions(+) diff --git a/docs/ARCHITECTURAL-DECISIONS-2026-06-04.md b/docs/ARCHITECTURAL-DECISIONS-2026-06-04.md index 8adfd89..7e519da 100644 --- a/docs/ARCHITECTURAL-DECISIONS-2026-06-04.md +++ b/docs/ARCHITECTURAL-DECISIONS-2026-06-04.md @@ -51,6 +51,7 @@ | ADR-021 | **Meta-hygiene**: always grep peer crates before copying manifest patterns (the `[lints] workspace = true` cascade lesson) | **Pinned** | OGAR PR #15 + PR #17/#18 follow-ups | | ADR-022 | **The Firewall** — absolute inner/outer boundary; no serialization in hot path; inner = compile-time HHTL; outer = contract-trait pluggable | **Pinned** | OGAR (this PR); `docs/THE-FIREWALL.md` | | ADR-023 | **IR-as-wire-truth** — the source-language AST is *input dialect*; the canonical `Class`/`Attribute`/`Association`/`EnumDecl`/`ActionDef` IR is *wire truth*. Adapters lift dialects into IR; the IR routes everything (registry key, actor mailbox, Lance version, audit-log dimension) | **Pinned** | OGAR (this PR); `crates/ogar-vocab/`; `bardioc/substrate-b-shadow::EdgeDecoder` (PR #19) | +| ADR-024 | **Palette256 + HHTL codec** — the substrate's universal compression primitive. HHTL prefix establishes a frame; within the frame, values cluster; clustered values quantize to 256-index palette + const-table lookup. Names an existing primitive (Binary16K perms + bgz-tensor attention + arm-discovery aerial codebook, ρ=0.9973 vs cosine) rather than proposing one | **Pinned** | OGAR (this PR); `MedCare-rs/crates/medcare-analytics/src/{graph_contract.rs,column_mask_bridge.rs}`; `bgz-tensor/examples/compare_stacked_vs_i16.rs`; `lance-graph-arm-discovery` | ## ADR-001: `State = ActionState` (lifecycle), not domain state, for Rubicon binding @@ -1167,6 +1168,185 @@ lance-graph) before merge. - `docs/RDF-OWL-ALIGNMENT.md` §3 (OGAR's position in L1-L5) — the IR sits at the AR-pattern lift seam. +## ADR-024: Palette256 + HHTL codec — the substrate's universal compression primitive + +**Status:** Pinned (2026-06-05). Names an existing primitive (three +independent deployments + one empirical anchor) rather than proposing +one. Companion to ADR-022 (The Firewall — this ADR specifies one of +its inner-side primitives) and ADR-023 (IR-as-wire-truth — palette256 +is the codec on the IR's wire form). + +**Context.** The substrate has accumulated three independent +palette256 deployments developed for their own domains: + +- **Security mesh** — `Binary16K = [u64; 256]` in + `MedCare-rs/crates/medcare-analytics/src/graph_contract.rs:36` + (canonical home). The per-row `_effectiveReaders` bitmap; auth is + Hamming-popcount bit-intersection at the inner / hot path + (`HEALTHCARE-TRANSCODING.md §3.1`). Wired into production at + `MedCare-rs/crates/medcare-analytics/src/column_mask_bridge.rs` → + `medcare-server/state.rs:167, 265, 439`. +- **Attention** — `bgz-tensor` `WeightPalette::build(…, 256)` + + `AttentionTable::build` (`crates/bgz-tensor/examples/ + compare_stacked_vs_i16.rs:90-92`). Replaces dense FP weights with + 256-index palette + precomputed distance table on the model's hot + path. +- **Distance** — `lance-graph-arm-discovery` aerial codebook — + measured **ρ = 0.9973 vs cosine**. The empirical anchor: palette256 + reproduces cosine distance with correlation 0.9973 (i.e. on a + scale where 1.0 = identical, palette256 is ~0.003 from cosine). + +Cross-domain analysis revealed all three are instances of the *same +codec*: HHTL prefix establishes a frame; within frame, values +cluster; clustered values quantize to a 256-index palette; decode is +a const-table lookup. The runtime side's BindSpace dissolution work +(bardioc PR #18 / lance-graph PR #470) hinted at this with the +Quintenzirkel qualia codebook ("frozen set + circle-of-fifths +progression → 8 B → 1-2 B per row") — same compression strategy, +different domain. + +The proposal in the cross-session conversation (2026-06-05) was to +name the primitive explicitly so: +1. Future adopters don't reinvent it per domain. +2. New adopters report a falsifiable measurement (ρ-vs-reference) + at adoption time rather than after the fact. +3. The 256-ceiling escape hatches are documented before reviewers + ask. + +**Decision.** **The codec is:** + +```text +HHTL prefix (NiblePath / quadkey / class identity) + ↓ establishes spatial / semantic frame +within-frame values cluster + ↓ quantize to 256-index palette + ↓ const-table lookup (compile-time HHTL where possible) +1-byte index per element, sub-microsecond decode, zero heap allocation +``` + +**Adoption checklist** for a new domain: +1. **Identify the prefix.** The NiblePath / quadkey / class identity + that establishes the frame the values live in. +2. **Identify the palette domain.** Which values cluster within the + frame? (Closed-keyspace tags, quantized continuous values, + enumerated state, etc.) +3. **Build the palette + measure ρ-vs-reference.** The reference is + the domain's full-precision metric (cosine for embeddings, L2 for + coordinates, exact-match for tags). Report ρ at adoption time as + the falsifiable property. Target: **ρ ≥ 0.99** to match the + arm-discovery anchor. +4. **Decode = const-table lookup.** Compile-time HHTL if the palette + is static; runtime const-table if the palette is per-frame / + per-tile. Either way the decode path is zero-allocation. + +**The 256-ceiling escape hatches** (documented to avoid the +predictable reviewer question): + +- **Per-tile / per-frame palettes** — the cheapest answer. Different + spatial-frame, different 256 entries. Used by Cesium tile codecs; + matches the quadkey-prefix discipline. Long-tail OSM tags inside a + zoom-21 tile rarely exceed 256. +- **Hierarchical palettes** — coarser palette at higher quadkey + levels, finer per leaf. Mirrors the standard tile pyramid; the SH + L0/L1 vs L2/L3 split in `splat-fit` is the same pattern. +- **Palette-64K upgrade** — 2-byte index instead of 1, for hot + palettes that genuinely exceed 256 distinct values (rare; reserve + for measured cases, not speculation). + +The escape hatches are part of the primitive, not exceptions to it. + +**Alternatives considered.** + +- *Continuous distributions that don't cluster* (e.g. timestamps in + microseconds, free-form text). Rejected as a counterargument to + the codec — these are out-of-domain. For them, use delta encoding + or VarInt or a different codec entirely. The codec applies to + *clustered* domains; the adoption checklist's step 2 is the filter. +- *Domain-specific codecs per domain.* Rejected. Three independent + re-derivations of the same primitive (security / attention / + distance) is the receipt that the abstraction is real, not the + receipt that each domain should have its own. ADR-024 reduces + per-domain re-derivation. +- *Skip the ρ-vs-reference measurement.* Rejected. The arm-discovery + ρ = 0.9973 is the existing FINDING-grade stake; new domains + reporting at adoption time keeps the empirical floor honest as the + primitive spreads. + +**Consequences.** + +- **The primitive is named.** Cross-domain reuse is now load-bearing, + not coincidental. New domains adopt the codec instead of inventing + their own quantization. +- **ρ-vs-reference becomes the adoption contract.** Reported once at + adoption per domain. The arm-discovery 0.9973 is the existing + anchor; new adopters target ≥ 0.99 and document if they fall short. +- **Two next-domain adopters are queued** (planned, not yet wired): + - **D-OSM-2** (OSM tag palette + tile-local coordinate + quantization) — per `lance-graph` PR #473 (`cesium-osm-substrate + -v1.md`). Reports ρ-vs-reference on first per-country PBF run per + the runtime session's §11 follow-up commitment. + - **D-SPLAT-4** (SH-aware palette extension on the + `Gaussian3D` carrier) — per the splat-native arc. Same codec; SH + coefficients are the long-tail-budget challenger. +- **The 256-ceiling has three explicit escapes** in the ADR body + (per-tile / hierarchical / palette-64K). Reviewers don't need to + re-derive the answer. +- **Cross-arc reuse argument is sharpened.** The substrate-reuse + framing in `docs/RDF-OWL-ALIGNMENT.md §10` (geographic litmus + complements anatomical) cashes out as: FMA-bones and OSM-vectors + use *the same codec* (palette256 + HHTL prefix), not just the same + IR. The §6 callout in `DOMAIN-INSTANCES.md` (queued, awaiting + lance-graph PR #473 land) will reference ADR-024 as the falsifiable + property. +- **The falsifiable property** that ties the substrate-reuse claim + down: *"the same compile-time HHTL prefix + palette256 codec + decodes (a) `_effectiveReaders` for row auth, (b) OSM way + attributes at zoom-21 tile, and (c) FMA-bone SH coefficients at + sub-microsecond per element with zero heap allocation."* If that + property holds across all three, the substrate is doing its job. + If it fails on one, the substrate is leaking dialect into the codec. + +**Change policy.** Adding a new palette256 adopter (new domain) is +routine — follow the adoption checklist + report ρ-vs-reference. +Changing the codec itself (e.g. palette-64K becoming default, or a +new escape-hatch added) is a substrate-wide concern and requires +consultation with the runtime session. + +**References.** + +- `lance-graph/.claude/board/EPIPHANIES.md:28` — FINDING-grade + anchor for palette256 + Hamming popcount on `_effectiveReaders`. +- `lance-graph/.claude/knowledge/old-stack-capability-parity.md §3.39` + — knowledge-doc record of the same primitive. +- `MedCare-rs/crates/medcare-analytics/src/graph_contract.rs:36` — + `Binary16K = [u64; 256]` canonical home. +- `MedCare-rs/crates/medcare-analytics/src/column_mask_bridge.rs` — + production wire-up; `redaction_mode_for` (line 128), + `column_mask_policy_for_table` (line 165), + `build_medcare_column_mask_registry` (line 192). +- `MedCare-rs/crates/medcare-server/src/state.rs:167, 265, 439` — + F2-E install sites consuming the column-mask registry. +- `bgz-tensor/examples/compare_stacked_vs_i16.rs:90-92` — + `WeightPalette::build(…, 256)` + `AttentionTable::build`. +- `lance-graph-arm-discovery` — aerial codebook with ρ = 0.9973 vs + cosine measurement. +- ADR-022 (The Firewall) — the inner-side discipline this ADR + specifies a primitive for. +- ADR-023 (IR-as-wire-truth) — palette256 is the codec on the IR's + wire form. +- `docs/THE-FIREWALL.md` §3 (the inner/hot side) — palette256 + HHTL + is one of its load-bearing primitives. +- `docs/HEALTHCARE-TRANSCODING.md §3.1` — palette256 + Hamming + popcount on Binary16K named as the inner-side security mesh. +- `docs/RDF-OWL-ALIGNMENT.md §10` — the brutal-upgrade sequencing + context (Phase 2c geospatial adopts the codec). +- `bardioc` PR #18 + `lance-graph` PR #470 — Quintenzirkel qualia + codebook (8 B → 1-2 B per row) as the same compression strategy in + a different domain. +- `lance-graph` PR #473 (forthcoming) `cesium-osm-substrate-v1.md` + §11 — runtime-side commitment to a follow-up callout on this ADR + once D-OSM-2 / D-SPLAT-4 wire. + ## Implementation receipts — ADR ↔ commit cross-reference > **Added in follow-up addendum (2026-06-05).** Records the implementation