feat(arm-discovery): D-ARM-14 Phase 1 — splat-top-k oracle + DOLCE skeleton projector

.claude/board/AGENT_LOG.md

@@ -1,3 +1,13 @@
+## [Main thread / Opus] D-ARM-14 Phase 1 — splat-top-k oracle + DOLCE skeleton projector
+
+**Branch:** claude/jolly-cori-clnf9-darm14 (off post-merge main) | **Files:** `crates/lance-graph-arm-discovery/src/{aerial/codebook.rs (+TopKDistance), aerial/ontology.rs (new), aerial/mod.rs, encode.rs (+checked_slot), lib.rs}` + STATUS_BOARD (D-ARM-14 → In progress).
+
+**Cargo:** **41/41** (37 + 4 new) + clippy `-D warnings` clean on BOTH default and `--features ndarray-simd`. Zero-dep preserved.
+
+**Outcome:** Phase 1 DONE. User chose D-ARM-14 next. Recon (consult-before-guess): `crates/lance-graph-ontology` (DOLCE hydrators) + the blasgraph splat top-k both live in heavy workspace crates; `jc::ewa_sandwich` is a PROOF not a table-builder — none buildable from the zero-dep aerial crate. So Phase 1 = the two **aerial-side seams**, verifiable standalone: (1) `TopKDistance` — a sparse per-node top-k `CodebookDistance` (the real shape the 10000² Gaussian-splat emits; you keep top-k per node, certified by jc EWA-sandwich, not a dense dim² table), symmetric, nearest-on-duplicate, bounds-checked via new `FeatureSpec::checked_slot`; (2) `aerial::ontology` — `DolceCategory` (Endurant/Perdurant/Quality/Abstract + basin nibble + IRI, the HHTL axis template) + `OntologyProjector` (FeedProjector → `rdfs:subClassOf`/`rdf:type` DOLCE skeleton SPO). End-to-end test proves splat-top-k → aerial discovers `occupation→DOLCE-class` → projects `wd:f0_0 rdfs:subClassOf dolce:Endurant`, and an unused facet is never invented. Float stays OFFLINE in jc only; aerial online path integer (CAM-PQ doctrine). Remaining (documented): real jc/blasgraph splat producing the lists, Wikidata loader, D-ARM-7 Jirak floor. NOT pushed yet — awaiting branch confirmation (prior branch merged). See `splat-codebook-aerial-wikidata-compression.md`.
+
+---
+
 ## [Main thread / Opus] PR #436 review response — 5 CodeRabbit findings (public-API hardening)
 
 **Branch:** claude/jolly-cori-clnf9 | **Files:** `crates/lance-graph-arm-discovery/src/{aerial/codebook.rs, aerial/extract.rs, translator.rs, encode.rs}` + `.claude/knowledge/aerial-arm-ruff-spo-codegen-synergies.md`.

.claude/board/STATUS_BOARD.md

@@ -635,7 +635,7 @@ The missing UPSTREAM discovery leg. Today's proposers (curated L-docs + AST-extr
 | D-ARM-11 | `style_recipe.rs` rule 8 — ArmDiscovered backing adds `DAtom::Compute` weight 2 (provisional) | lance-graph-ontology::style_recipe | 80 | MED | **Queued** | depends on D-ARM-1 |
 | D-ARM-12 | End-to-end pipeline test + bench (synthetic Odoo feed → all 5 stages → council micro-batch) | lance-graph-arm-discovery::tests + benches | 400 | MED | **Queued** | depends on Waves 1-6; informs OQ-ARM-2 + OQ-ARM-7 |
 | D-ARM-13 | **Aerial+ Rust transcode — deterministic codebook-probe backend** (float-free). The paper's `f32` denoising autoencoder is REPLACED by an integer `CodebookDistance` oracle (palette256 distance, ρ=0.9973 vs cosine): the reconstruction probe is a codebook top-k, not a softmax over float weights. Integer evidence counts + ppm gates + `TruthU8` (= CausalEdge64 wire). `AerialProposer` impl of `Proposer`. Count loop is a row-bitset SoA (`RowMasks`) → AND+popcount, routed through `ndarray::simd::U64x8` under the `ndarray-simd` feature. | lance-graph-arm-discovery::aerial | ~1.1K | HIGH | **Shipped (branch)** | branch `claude/jolly-cori-clnf9`; standalone zero-dep crate (excluded); **33/33** tests + clippy `-D warnings` clean on BOTH default (scalar) and `--features ndarray-simd`; **zero f32 in the discovery path** (audit), float only at the `TruthValue`/`Triple` serialization edge. Bitwise-deterministic ⇒ joins the trunk; the nondeterminism firewall + D-ARM-9 IPC rationale are moot. SIMD target-cpu caveat: real AVX-512/AMX kernels need `-C target-cpu=native`/`x86-64-v4`. v1 (autoencoder) superseded per the user's no-float directive. |
-| D-ARM-14 | **Splat-codebook oracle + Wikidata skeleton discovery** — wire the certified jc splat codebook into aerial as the `CodebookDistance` oracle, discover OWL/DOLCE+ SPO HHTL classes + basins, drive the `wikidata-hhtl-load.md` deterministic compression (skeleton + basins + CAM-dedup + thin rows). | lance-graph-arm-discovery::aerial + crates/jc + wikidata loader | ~? | MED | **Queued (architecture)** | CONJECTURE pipeline; seams concrete (`CodebookDistance` ← jc `[u32;dim²]` table built by `ewa_sandwich`+`sigma_codebook_probe` ρ=0.9973+`pflug` Lε; Jirak floor ← `jc::jirak`). Float lives OFFLINE in jc only; aerial online path integer. No new aerial dep (pass the frozen table in). Gated on D-ARM-7. Full map: `splat-codebook-aerial-wikidata-compression.md`; finding E-ARM-JC-RESOLVES-BOTH-SEAMS. |
+| D-ARM-14 | **Splat-codebook oracle + Wikidata skeleton discovery** — wire the certified jc splat codebook into aerial as the `CodebookDistance` oracle, discover OWL/DOLCE+ SPO HHTL classes + basins, drive the `wikidata-hhtl-load.md` deterministic compression (skeleton + basins + CAM-dedup + thin rows). | lance-graph-arm-discovery::aerial + crates/jc + wikidata loader | ~? | MED | **In progress** | **Phase 1 (branch `claude/jolly-cori-clnf9-darm14`):** the two aerial-side seams — `aerial::TopKDistance` (the sparse splat-top-k `CodebookDistance` the 10000² BLASGraph splat actually emits; top-k per node, not a dense table) + `aerial::ontology::{DolceCategory, OntologyProjector}` (DOLCE 4-facet skeleton → `rdfs:subClassOf`/`rdf:type` SPO). End-to-end test: splat top-k → aerial discovers `occupation→DOLCE-class` → projects the skeleton triple. 41/41 + clippy clean (default + `ndarray-simd`), zero-dep. Float still OFFLINE in jc only (`ewa_sandwich`+`sigma_codebook_probe` ρ=0.9973+`pflug` Lε); aerial online path integer. **Remaining:** real jc/blasgraph splat producing the lists; Wikidata loader; gated on D-ARM-7 (`jc::jirak`). Map: `splat-codebook-aerial-wikidata-compression.md`; E-ARM-JC-RESOLVES-BOTH-SEAMS. |
 | D-ARM-SYN-1 | Add `Implies`/`CoOccursWith` to `ruff_spo_triplet::Predicate` closed vocabulary (+ `Provenance` tier) so ARM rules load through the same `parse_triples` ndjson path as the static extractor | ruff/ruff_spo_triplet | 40 | MED | **Queued** | council-gated (deliberate ontology change); blocks SYN-2; see `.claude/knowledge/aerial-arm-ruff-spo-codegen-synergies.md` §1 |
 | D-ARM-SYN-2 | `CandidateRule → ruff_spo_triplet::ModelGraph` adapter so the Aerial runtime-data leg joins the `ruff_python_dto_check` static-AST leg in one graph before `expand()` | lance-graph-arm-discovery + ruff_spo_triplet | 120 | MED | **Queued** | depends on SYN-1; synergy doc §2 |
 | D-ARM-SYN-3 | Calibrate `ProvenanceTier::ArmDiscovered` `(f,c)` below the `op_emitter` ratification gate + below static `Inferred (0.85,0.75)` so un-ratified ARM truth is council-visible but codegen-filtered | lance-graph-contract + lance-graph-ontology::op_emitter | 30 | MED | **Queued** | depends on D-ARM-1 + SYN-1; synergy doc §3/§4 |

.claude/knowledge/splat-codebook-aerial-wikidata-compression.md

@@ -94,7 +94,8 @@ codebook offline (float OK), address it online with integer codes.** jc is the
 
 ## Status & open questions
 
-- **CONJECTURE** (architecture): the splat→aerial→Wikidata wiring as a whole. The *seams* are concrete (the `CodebookDistance` trait + `jc::jirak`); the end-to-end pipeline is not yet built.
+- **Phase 1 SHIPPED** (D-ARM-14, branch `claude/jolly-cori-clnf9-darm14`): the two aerial-side seams are concrete code, not just a trait — `aerial::TopKDistance` (the sparse per-node splat-top-k `CodebookDistance` the 10000² splat actually emits — top-k per node, not a dense `dim²` table) and `aerial::ontology::{DolceCategory, OntologyProjector}` (the DOLCE 4-facet skeleton → `rdfs:subClassOf`/`rdf:type` SPO output). An end-to-end test runs splat-top-k → discovery → skeleton projection. Standalone, integer, 41/41.
+- **CONJECTURE** (architecture): the splat→aerial→Wikidata pipeline *as a whole* — the real jc/blasgraph splat producing the neighbour lists, and the Wikidata loader, are not yet built. The seams they plug into now exist.
 - **Hard prerequisite:** D-ARM-7 (the Jirak floor) must land before aerial promotes any rule to a live skeleton — ISSUE ARM-JIRAK-FLOOR. jc::jirak is the engine; the *gate function* (rule → significant?) is the deliverable.
 - **No new aerial dependency needed:** jc emits a frozen integer table; aerial consumes it through the existing `MatrixDistance`/`CodebookDistance` seam. Do NOT make aerial depend on jc — keep the float-free standalone posture; pass the certified table in.
 - **Open (from wikidata-hhtl-load):** P279 fan-out is wildly uneven (2…4000 children) — whether it rebalances onto a clean 16ⁿ tree or forces adaptive fan-out is MEASURABLE; measure on a real P279 subtree before fixing the HHTL base.

crates/lance-graph-arm-discovery/src/aerial/codebook.rs

@@ -88,6 +88,64 @@ impl CodebookDistance for MatrixDistance {
     }
 }
 
+/// A [`CodebookDistance`] backed by a **sparse top-k neighbour list per item** —
+/// the shape the BLASGraph Gaussian-splat spatial top-k (10000²) actually
+/// emits. You keep only the `k` nearest neighbours per node (certified by jc's
+/// EWA-sandwich Σ-push-forward), never a dense `dim²` table. `distance(a, b)`
+/// returns the stored splat distance if `b ∈ topk(a)`, else `miss` (far).
+///
+/// This is the production oracle shape for D-ARM-14 (the [`MatrixDistance`]
+/// dense table is the test/reference impl). The splat that *builds* the
+/// neighbour lists is the consumer-side `crates/jc` / blasgraph step; this crate
+/// stays zero-dep and just consumes the frozen integer lists.
+#[derive(Debug, Clone)]
+pub struct TopKDistance {
+    spec: FeatureSpec,
+    miss: u32,
+    /// `neighbours[slot]` = ascending `(neighbour_slot, distance)`, dedup'd to
+    /// the nearest distance per neighbour. Binary-searchable.
+    neighbours: Vec<Vec<(u32, u32)>>,
+}
+
+impl TopKDistance {
+    /// Build from undirected splat edges `(a, b, distance)`. `miss` is the
+    /// distance returned for pairs not in each other's top-k (i.e. "far").
+    #[must_use]
+    pub fn new(spec: FeatureSpec, miss: u32, edges: &[(Item, Item, u32)]) -> Self {
+        let dim = spec.dim();
+        let mut neighbours: Vec<Vec<(u32, u32)>> = vec![Vec::new(); dim];
+        for &(a, b, d) in edges {
+            let (ca, cb) = (spec.checked_slot(a), spec.checked_slot(b));
+            neighbours[ca].push((cb as u32, d));
+            neighbours[cb].push((ca as u32, d)); // splat distance is symmetric
+        }
+        for list in &mut neighbours {
+            // sort by (neighbour, distance) so dedup_by_key keeps the nearest.
+            list.sort_unstable();
+            list.dedup_by_key(|&mut (n, _)| n);
+        }
+        Self {
+            spec,
+            miss,
+            neighbours,
+        }
+    }
+}
+
+impl CodebookDistance for TopKDistance {
+    fn distance(&self, a: Item, b: Item) -> u32 {
+        let ca = self.spec.checked_slot(a);
+        let cb = self.spec.checked_slot(b) as u32;
+        if ca as u32 == cb {
+            return 0;
+        }
+        match self.neighbours[ca].binary_search_by_key(&cb, |&(n, _)| n) {
+            Ok(i) => self.neighbours[ca][i].1,
+            Err(_) => self.miss,
+        }
+    }
+}
+
 /// Aggregate the distance from a (multi-item) antecedent to a candidate
 /// consequent item: the **nearest** antecedent item wins (min). For a
 /// single-item antecedent this is just `distance(a, item)`.
@@ -149,4 +207,27 @@ mod tests {
         // category 5 ≥ feature-0 block width 2 — must fail, not alias feature 1.
         let _ = d.distance(Item::new(0, 5), Item::new(1, 0));
     }
+
+    #[test]
+    fn topk_distance_is_sparse_and_symmetric() {
+        let spec = FeatureSpec::new(vec![2, 2]);
+        // one splat edge (f0,c0)~(f1,c0) at distance 3; all else "far" (99).
+        let edges = [(Item::new(0, 0), Item::new(1, 0), 3)];
+        let d = TopKDistance::new(spec, 99, &edges);
+        assert_eq!(d.distance(Item::new(0, 0), Item::new(1, 0)), 3);
+        assert_eq!(d.distance(Item::new(1, 0), Item::new(0, 0)), 3, "symmetric");
+        assert_eq!(d.distance(Item::new(0, 0), Item::new(1, 1)), 99, "non-neighbour = far");
+        assert_eq!(d.distance(Item::new(0, 0), Item::new(0, 0)), 0, "self = 0");
+    }
+
+    #[test]
+    fn topk_keeps_the_nearest_on_duplicate_edges() {
+        let spec = FeatureSpec::new(vec![2, 2]);
+        let edges = [
+            (Item::new(0, 0), Item::new(1, 0), 9),
+            (Item::new(0, 0), Item::new(1, 0), 2), // nearer duplicate wins
+        ];
+        let d = TopKDistance::new(spec, 99, &edges);
+        assert_eq!(d.distance(Item::new(0, 0), Item::new(1, 0)), 2);
+    }
 }

crates/lance-graph-arm-discovery/src/aerial/mod.rs

@@ -29,9 +29,11 @@
 
 pub mod codebook;
 pub mod extract;
+pub mod ontology;
 
-pub use codebook::{antecedent_distance, CodebookDistance, MatrixDistance};
+pub use codebook::{antecedent_distance, CodebookDistance, MatrixDistance, TopKDistance};
 pub use extract::{extract_rules, ExtractParams};
+pub use ontology::{DolceCategory, OntologyProjector};
 
 use crate::encode::Dataset;
 use crate::rule::{CandidateRule, Proposer};