From 1f5e09553edcdbf1f5a855a28092879471b789cd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 13:59:51 +0000 Subject: [PATCH 1/9] plans: transcode EXTEND-CORE council record + re-scoped C-FIRST decision Records the full 5-consolidate + 3-brutal-critique council on the Tesseract C++->Rust transcode next-move decision. 5 consolidation agents: core-first-architect (TARGETS-CORE, found the ocr.rs classid-keyed-registry precedent), container-architect (ADDITIVE-CONFIRMED, zero locked-node impact), adapter-shaper (THIN-CONFIRMED, scoped to old_style_included_), truth-architect (PREMATURE), integration-lead (SEQUENCE C->A->D->B). 3 brutal critics converge: the original "self-authored golden + run-twice determinism" gate is a tautology (truth-architect, brutally-honest-tester, adk-behavior-monitor all independently). Replacement gate, named by all three: a round-trip structural-equivalence falsifier (expand -> ndjson -> from_ndjson -> reassemble, assert ~= original) that is immune to the harvest-freshness drift (live 2032 triples vs committed 880) and CAN fail (real UNICHARMAP vs UNICHARSET unichar_to_id collision). baton-handoff-auditor: CATCH-LATENT, no CATCH-CRITICAL; bakes in the design constraint that reassembly derives per-overload identity from the index-prefixed has_param_type triples, never the (params) IRI suffix (no clean inverse for comma-bearing templated types). Final 8/8 decision: execute re-scoped C-FIRST. OCR-SCHEMA mis-cite dropped; Frankenstein-refusal becomes an honest deny-list test; PARITY: UNRUN honesty markers required; byte-parity promotion stays operator-gated. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 .claude/plans/transcode-extend-core-probe-v1.md diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md new file mode 100644 index 00000000..8bc2784e --- /dev/null +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -0,0 +1,164 @@ +# Transcode EXTEND-CORE + in-env probe slice — v1 (UNDER COUNCIL REVIEW) + +> **Status:** PROPOSAL — under 5-consolidate + 3-brutal council review (2026-06-17). +> NOT ratified. Touches operator-locked OGAR canon (additively); do not implement +> until the council lands a LAND verdict. +> **Decision owner:** operator delegated autonomous decision-making backed by the +> council (this session). The council IS the review the doctrine's EXTEND-CORE rule +> requires ("filed + reviewed"). + +## Context (what's shipped, what the design agents found) + +- **Harvester shipped** — `ruff_cpp_spo` (AdaWorldAPI/ruff PR #17, merged): 17 C++ + machine-plane SPO predicates, codegen-ready, deterministic. `tesseract::UNICHARSET` + harvested with full method signatures (`unichar_to_id`/`id_to_unichar` as + `has_function` + `returns_type` + ordered `has_param_type` + `is_const`, per-overload + `(params)` IRI suffix). +- **Codegen plan committed** — `tesseract-rs/.claude/plans/tesseract-rs-ast-dll-codegen-v1.md` + (the adapter-body half). Generator is standalone-buildable; byte-parity is gated on a + leptonica build env this checkout lacks. +- **Probe ruling (core-gap-auditor):** `EXTEND-CORE`. The unicharset id↔utf8 bijection is + a variable-length **shared registry**, NOT fixed-width per-node state — so it rides a + **classid-keyed content-store tier** (shaped exactly like the shipped + `deepnsm::Vocabulary`), resolved `classid → &UniCharSet` via a `LazyLock` registry + mirroring `classid_read_mode`/`BUILTIN_READ_MODES`. An adapter owning its own table = + the rejected Adapter-State-Leak. Byte-parity is BLOCKED (no leptonica; zero `.unicharset` + on disk). + +## The decision under review + +Implement the **EXTEND-CORE content-store tier + the in-env dispatch-validation probe +slice** in lance-graph now (Option A below). + +## Proposed change — ADDITIVE ONLY (the load-bearing safety claim) + +It does **NOT** modify the locked 512-byte node: no new `ValueTenant`, no `ValueSchema` +variant, no stride change, no `ENVELOPE_LAYOUT_VERSION` bump. It adds a **sibling content- +store tier** — the doctrine's `I-VSA-IDENTITIES` Layer-3 content store, the shape the Core +"already has." + +1. **`UniCharSet` content store** — `reverse: Vec` (id→repr, byte-exact mirror of + C++ `unichars[id].representation`) + `lookup: HashMap` (the `UNICHARMAP` + inverse). `id_to_unichar(id)->&str`, `unichar_to_id(repr)->u32` (sentinel on miss). + Structurally one-for-one with `deepnsm::Vocabulary`. +2. **`classid → &UniCharSet` resolver** — `LazyLock>` + free fn, + structurally identical to `classid_read_mode`. +3. **Mint one non-default `UNICHARSET` classid.** +4. **Two thin adapters** — `unichar_to_id`/`id_to_unichar`, each a lookup, zero owned state. + (For the probe scope use the `old_style_included_ == true` semantics so the leaf is a + pure table lookup; the `CleanupString` normalization is a separate, later leaf.) +5. **ClassView composing them** from the harvested `has_function` manifest; invoke via + `UnifiedStep`. +6. **Dispatch-validation test** — round-trip bijection (`id_to_unichar(unichar_to_id(s)) == s`, + and `unichar_to_id() == INVALID`) against a **hand-authored `.unicharset`** in + the documented text format — no libtesseract. + +## What it proves / does NOT prove + +- **Proves:** the Core CAN hold the adapters end-to-end — content store + classid + + resolver + ClassView-composed-from-manifest + `UnifiedStep` dispatch. A leak here would + be the cheapest possible Core-gap discovery (the auditor expects none — the registry tier + fits). +- **Does NOT prove:** byte-parity with libtesseract on a real trained table. So it CANNOT + promote the doctrine CONJECTURE→FINDING, and **scaling stays BLOCKED** per the iron guard. + +## Alternatives the council must weigh + +- **A — EXTEND-CORE probe slice now (this plan).** Validates the mechanism in-env; + additive; sets up the codegen. Does not reach byte-parity. +- **B — Golden-fixture first.** Defer EXTEND-CORE until real byte-parity is possible. + Requires the operator to run a leptonica host to capture `eng.unicharset` + expected-ids. + Only path to FINDING, but blocks all in-env progress on the operator's host. +- **C — Codegen-generator standalone first.** Build the ndjson→Rust generator + its tests + (reassembly/signature/golden/Frankenstein-refusal), no Core change, defer the probe. +- **D — Harvester-polish first.** GAP-3 `has_visibility` (genuinely-dropped access + specifiers). Orthogonal; small; verifiable in ruff. (Note: codegen's GAP-1 `member_of` + is REDUNDANT — member→class is recoverable by inverting `has_function`/`has_field`; it is + a codegen-side fix, not a harvester predicate.) + +## Council questions + +1. **core-first-architect:** does A TARGETS-CORE (vs residue / parallel-model)? Is the + content-store tier the right home, or is it smuggling a second object model? +2. **truth-architect:** is building A justified when byte-parity (the real validation) is + BLOCKED? Is the dispatch-validation a real measurement or synthesis-without-a-falsifier? +3. **container-architect:** is A genuinely ADDITIVE — zero impact on the locked + `NodeRow`/`ValueTenant`/`ValueSchema`/stride/layout-version? Any hidden lock break? +4. **integration-lead:** sequencing — A vs B vs C vs D. What does each unblock; what's the + critical path to FINDING + scaling? +5. **adapter-shaper:** are `unichar_to_id`/`id_to_unichar` genuinely thin mechanical leaves + under this shape, or does the real C++ (`CleanupString`, fragments) leak state into them? + +## Consolidation — 5-agent council (2026-06-17) + +| Agent | Verdict | Load-bearing finding | +|---|---|---| +| core-first-architect | **TARGETS-CORE** | The content-store tier targets the Core, doesn't treat it as residue. Found the `ocr.rs::LayoutBlock::to_node_row` precedent — a shipped classid-keyed adapter that already rides a variable-length registry beside the SoA node. Option A is the *same shape*, not a new object model. | +| container-architect | **ADDITIVE-CONFIRMED** | Zero impact on the locked `NodeRow`/`ValueTenant`/`ValueSchema`/stride/`ENVELOPE_LAYOUT_VERSION`. The content-store tier is a sibling `LazyLock` registry; no lock break, no hidden stride change. | +| adapter-shaper | **THIN-CONFIRMED (scoped)** | `unichar_to_id`/`id_to_unichar` are pure table lookups **iff** scoped to `old_style_included_ == true`. Precondition: fixture AND oracle must both be old-style; do NOT target `id_to_unichar_ext` (fragment-expansion leaks state). `CleanupString` normalization is a separate later leaf, correctly excluded. | +| truth-architect | **PREMATURE** | The Option-A dispatch-validation is a hand-authored bijection round-trip — a *tautology*. It re-derives the already-specced `ocr-probes-v1.md` OCR-SCHEMA without adding a falsifier that CAN fail. Building it now is synthesis-without-a-measurement. Byte-parity (the real validation) stays operator-BLOCKED. | +| integration-lead | **SEQUENCE C→A→D→B** | libclang-18 + the Tesseract corpus are present in-env → **Option C (the ndjson→Rust generator) is the only critical-path move fully runnable here, today.** A needs the council gate (touches canon); B needs the operator's leptonica host; D is orthogonal polish. C unblocks A and B both. | + +### Consolidated leading decision: **C-FIRST** + +Revises the plan's original "Option A now". The council splits cleanly: A is +*architecturally sound* (3 of 5 confirm shape/additivity/thinness) but the *wrong +next move* (2 of 5: it's a tautology gated behind canon-review, while the +critical-path in-env work is unblocked and waiting). + +**The C-FIRST move, verbatim (to be brutally critiqued before execution):** + +1. **C — build + test the codegen generator (ndjson→Rust)** against REAL harvested + `tesseract::UNICHARSET` ndjson, fully in-env: reassembly (anchor-first group-by), + signature reconstruction from `returns_type`/`has_param_type`/`is_const`, + overload-split on the `(params)` IRI suffix, golden + run-twice determinism, + Frankenstein-refusal (intrusive/stateful method → route to hand-port, never force + the adapter mold). **No Core change → no council gate.** +2. **OCR-SCHEMA** — run the already-specced, free, in-env falsifier that CAN fail + (`ruff/.claude/plans/ocr-probes-v1.md`), satisfying truth-architect's + measurement-before-synthesis demand. +3. **A — EXTEND-CORE probe slice** — AFTER C lands and the brutal council returns + LAND. Still additive, still council-gated. +4. **B — golden fixture vs libtesseract** — handed to the operator (leptonica host); + the only path to CONJECTURE→FINDING. + +## Brutal panel (3 critics, 2026-06-17) — C-FIRST is RE-SCOPED, not rejected + +| Critic | Verdict | Load-bearing finding | +|---|---|---| +| brutally-honest-tester | **HOLD → re-scope** | Ran the real harvest. Of C-FIRST's 5 sub-tests, **2 are tautologies** (run-twice determinism is a pure-fn-over-sorted-input no-op already covered by `cpp_ast_rt_determinism`; the self-authored golden matches itself), **1 is impossible on current data** (Frankenstein-refusal: `id_to_unichar` and the state-leaking `id_to_unichar_ext` are triple-identical except for name — no closed-vocab predicate distinguishes table-lookup from fragment-expansion), and **2 have teeth** (reassembly, signature reconstruction). Real falsifier: the harvest contains `UNICHARMAP::unichar_to_id` AND `UNICHARSET::unichar_to_id` + two `UNICHARSET::unichar_to_id` overloads → correct reassembly must yield 4 distinct adapters. Also: **freshness landmine** (live harvest 2032 triples vs committed 880 — vocab still moving), and `ruff_cpp_spo::extract` is still `todo!()` (only the example path runs). | +| adk-behavior-monitor | **AP-flagged, borderline** | **OCR-SCHEMA is mis-cited** as the C-FIRST step-2 falsifier — it tests `ValueSchema` enum-fit on the Core side, zero connection to the harvest→generator chain, so it cannot satisfy the measurement demand. The self-authored golden is AP1 confirmation bias. Redirect (identical to the tester's): gate on the **round-trip against the harvester's own IR** (`CppClass → expand → ndjson → reassemble → assert ≈ original`); never let the generator author write the expected output. Honesty guardrail: every generated adapter carries `PARITY: UNRUN (operator-blocked: leptonica)` so a green generator suite is never misread as the byte-parity FINDING. | +| baton-handoff-auditor | **CATCH-LATENT** (no CATCH-CRITICAL) | Input contract is sound to build against: ndjson roundtrip is lossless (proven on real ccutil), `CppClass→Model` unpack is an exhaustive compile-checked match, reassembly anchor-first group-by is CLEAN (GAP-1 `member_of` confirmed a misdiagnosis), generated-Rust→Core names real Core types (no parallel model), content-store-vs-value-tenant correctly drawn. **Two P1 latent drops to queue:** (a) the `(params)` IRI suffix has **no clean inverse** for comma-bearing templated types (`std::map` → `f(std::map,int)` → naive `,`-split gives wrong arity) — **fix: derive overload identity from index-prefixed `has_param_type` triples, never parse the suffix**; (b) `virtually_overrides` target is string-reconstructed independently of the member IRI — **fix: harvester-side intra-graph referential-integrity test** (every override object must byte-equal an existing member-subject IRI). | + +### FINAL DECISION (8/8 consolidated): execute **re-scoped C-FIRST** + +Build the generator now (the in-env critical path), but with the gate replaced and +the tautologies/mis-cites dropped: + +1. **Gate = round-trip structural-equivalence falsifier, NOT a self-authored golden.** + The first executable deliverable is the **reassembler** (generator stage 1) + + a property test in `ruff_spo_triplet`: `ModelGraph → expand → Vec → + to_ndjson → from_ndjson → reassemble → ModelGraph'`, assert `ModelGraph' ≈ + ModelGraph`. This is the inverse of `expand()`, lives in the crate that owns both + directions, is **immune to the freshness landmine** (compares against the live + object), and **CAN fail** (the `UNICHARMAP`/`UNICHARSET` collision + overload + split are the adversarial cases). truth-architect's PREMATURE flag is resolved: + the round-trip is a real falsifier, not a tautology. +2. **Baton P1(a) baked in from line one:** reassembly derives per-overload identity + from the index-prefixed `has_param_type` triples, never by parsing the `(params)` + IRI suffix. (Avoids the comma-split latent drop before it can be written.) +3. **Drop run-twice as a gate** → replace with a `HashMap`-in-emit-path lint. + **Drop the OCR-SCHEMA mis-cite** from the C-FIRST step list. +4. **Frankenstein-refusal → honest hand-curated deny-list test** (assert + `id_to_unichar_ext` is hand-ported *by listing*, document the missing signal), + and **queue a harvester GAP** for an intrusiveness predicate (GAP-3 + `has_visibility` is the nearest candidate). +5. **Queue baton P1(b):** harvester intra-graph referential-integrity test for + `virtually_overrides` (before scaling past unicharset). +6. **Honesty markers:** when real `.rs` emission lands, every generated file + + `mod.rs` records `PARITY: UNRUN (operator-blocked: leptonica)`. A green generator + suite is NOT a green `PROBE-OGAR-ADAPTER-UNICHARSET`; byte-parity promotion lives + only at the operator's leptonica host (Option B). + +Sequence unchanged from integration-lead: **C (re-scoped) → A (EXTEND-CORE probe, +council-gated) → D (harvester polish) → B (byte-parity, operator)**. From 1eb30e20d40f4192e79fec1e50dcba54f39b4aae Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 14:17:08 +0000 Subject: [PATCH 2/9] =?UTF-8?q?plans:=20mark=20C-FIRST=20step=201=20LANDED?= =?UTF-8?q?=20=E2=80=94=20reassembler=20+=20real-corpus=20falsifier=20gree?= =?UTF-8?q?n?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The re-scoped C-FIRST gate is built in ruff: reassemble() (generator stage 1, inverse of expand) + the round-trip falsifier that replaced the council- rejected self-golden. CPP-REASSEMBLE-RT runs green on real Tesseract ccutil (67 classes; class-set preservation + idempotence). The falsifier found a real bug class — 19/67 const-overload IRI collisions (GAP-CONST-OVERLOAD, queued). truth-architect's PREMATURE flag resolved: real measurement on real data. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 33 +++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index 8bc2784e..5b6d51e4 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -162,3 +162,36 @@ the tautologies/mis-cites dropped: Sequence unchanged from integration-lead: **C (re-scoped) → A (EXTEND-CORE probe, council-gated) → D (harvester polish) → B (byte-parity, operator)**. + +## C-FIRST step 1 — LANDED (2026-06-17) + +The re-scoped gate is built and green (AdaWorldAPI/ruff, branch +`claude/happy-hamilton-0azlw4`): + +- **`ruff_spo_triplet::reassemble`** — generator stage 1, the inverse of + `expand`'s C++ projection. Method identity recovered from index-prefixed + `has_param_type` triples (baton P1(a) honored — never a `,`-split of the + `(params)` suffix); class attribution anchor-first. **`cpp_projection`** + exposed as the formal round-trip target. +- **Round-trip falsifier (NOT a self-golden):** in-crate, 6 tests incl. the + three adversarial cases (same-name-method cross-attribution guard, overload + split, comma-bearing templated param). 54 `ruff_spo_triplet` tests green. +- **`CPP-REASSEMBLE-RT` against the REAL corpus** (gated on `TESSERACT_SRC`): + Tesseract 5.5.0 ccutil, **67 classes — class-set preservation + idempotence + hold**. truth-architect's PREMATURE flag resolved: this is a real measurement + on real data, not a tautology. +- **Falsifier earned its keep — it found a real bug class:** 48/67 round-trip + byte-exact, **19 differ** = const/non-const overloads colliding on one method + IRI (const-ness absent from the `(params)` suffix; `expand`'s `(s,p,o)` dedup + merges them). Generalizes baton P1(a) to a cv-qualification axis, quantified + 19/67 on real data. Queued as **GAP-CONST-OVERLOAD** (harvester IRI-scheme + change, operator-gated) in `ruff/.claude/plans/cpp-spo-probes-v1.md` — a + documented known-gap, not a silent drop. +- **adk-behavior-monitor honesty guardrail** still owed when real `.rs` + emission lands (step ≥2): `PARITY: UNRUN (operator-blocked: leptonica)` + markers so a green generator suite is never misread as the byte-parity + FINDING (Option B, operator's host). + +**Next:** C step 2 (the Rust *emitter* consuming the reassembled `ModelGraph`) +remains; it is where the `PARITY: UNRUN` markers and the GAP-CONST-OVERLOAD +single-merged-adapter handling apply. Options A / D / B unchanged. From 256cc3438a8f5cc68c1c6ad8d56818da7a688dfd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 15:05:50 +0000 Subject: [PATCH 3/9] =?UTF-8?q?plans:=20C=20step=202=20(emitter)=20?= =?UTF-8?q?=E2=80=94=20plan=20+=205-agent=20consolidation?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records the step-2 emitter council. Major reframe (3/5 agents independently): the plan's premise was false — ClassView is a field/render vocabulary with NO method-resolution surface (has_function does not appear in lance-graph-contract). 5-agent consolidation: manifest-first cut (not stubs/bodies), additive (container ADDITIVE-CONFIRMED), placement = a new ruff_cpp_codegen crate that emits TEXT only (no ruff->lance-graph compile edge), D (const-overload fix) before the emitter, PARITY markers + Frankenstein deny-list. Unresolved fork handed to the 3-brutal panel: (A) mint a minimal MethodSig POD + classid_methods LazyLock registry (method-axis sibling of classid_read_mode, EXTEND-CORE but additive) vs (B) ride the SHIPPED codegen_spine::TripletProjection + roundtrip_eq (no new type; warns MethodSig re-implements CppMethod and that re-emitting the harvest is a tautology). codegen_spine.rs verified real (632 lines, TripletProjection trait + roundtrip_eq fn + Genericity enum). Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 112 ++++++++++++++++++ 1 file changed, 112 insertions(+) diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index 5b6d51e4..4496c73a 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -195,3 +195,115 @@ The re-scoped gate is built and green (AdaWorldAPI/ruff, branch **Next:** C step 2 (the Rust *emitter* consuming the reassembled `ModelGraph`) remains; it is where the `PARITY: UNRUN` markers and the GAP-CONST-OVERLOAD single-merged-adapter handling apply. Options A / D / B unchanged. + +--- + +## C step 2 — the emitter: plan + council (UNDER REVIEW 2026-06-17) + +> **Status:** PROPOSAL — under 5-consolidate + 3-brutal council review. The +> emitter is the first codegen that produces actual Rust *source*; the +> Core-First doctrine treats this as the central "parallel-model" danger, so it +> is gated by the doctrine ensemble before a line is written. + +### What step 2 is + +Stage 1 (`reassemble`, landed) turns the harvested triples back into a +`ModelGraph`. Stage 2 — the **emitter** — turns that `ModelGraph` into Rust +that targets the OGAR Core (`lance-graph-contract`: `canonical_node.rs` +NodeRow/classid, `class_view.rs` ClassView, `UnifiedStep`). The doctrine names +the harvest "the ClassView method-resolution manifest", so the emitter's job is +to project that manifest onto the Core — NOT to grow a second object model. + +### The fork the council must resolve — what does the emitter emit FIRST? + +The content-store tier for variable-length state (the unicharset id↔utf8 +registry) is **Option A, council-gated and NOT built**. So the emitter cannot +yet produce full unicharset adapter bodies. Three candidate first cuts: + +- **(1) Manifest-first (thinnest).** Emit ONLY the `classid → ClassView` + method-resolution manifest as Rust data: per class, the ordered method set + (name, params, return, const/static, override target) from `has_function` + + the signature predicates. No adapter bodies, no state mapping. Targets the + existing `ClassView` trait. Needs no content-store tier. Verifiable in-env + (compiles; manifest matches the harvest). Risk: too thin to be "codegen"? +- **(2) Signature-stub adapters.** Emit adapter fn signatures targeting Core + types, bodies = `// HAND-PORT` / `todo!()`. Forces the per-method + value-tenant vs content-store mapping decision now → hits the Option-A gate + for any unicharset leaf. Risk: blocked on Option A; premature. +- **(3) Full leaf-method bodies.** Deepest; needs the content-store tier + any + Core gaps resolved + byte-parity (operator-blocked). Out of scope for step 2. + +### Cross-cutting questions + +- **Placement:** does the emitter live in `ruff` (a `ruff_cpp_codegen` module + consuming `ruff_spo_triplet::ModelGraph`), in `tesseract-rs` (per + `tesseract-rs-ast-dll-codegen-v1.md`), or as a standalone? What does it + depend on, and does that respect the BBB / no-parallel-model rules? +- **Core-targeting:** does the emitted Rust reference REAL Core types + (`NodeRow`/`classid`/`ClassView`/`UnifiedStep`), or does it drift into a + parallel model? (doctrine's central litmus.) +- **GAP-CONST-OVERLOAD:** the 19/67 const-overload collisions — the emitter + must treat a colliding method IRI as a single merged adapter (documented), + not silently pick one. How is that surfaced in the generated output? +- **Honesty markers:** `PARITY: UNRUN (operator-blocked: leptonica)` on every + generated file + `mod.rs`. + +### Council questions + +1. **core-first-architect:** of the three first-cuts, which TARGETS-CORE vs + risks a PARALLEL-MODEL? Is manifest-first the right thinnest projection of + the "ClassView method-resolution manifest", or does even that smuggle a + second model? +2. **core-gap-auditor:** does the OGAR Core (today's `class_view.rs` ClassView + trait + `canonical_node.rs`) already accept the manifest the emitter would + produce, or is there a Core gap (EXTEND-CORE) before the emitter can target + it? Is manifest-first reachable without Option A? +3. **adapter-shaper:** for the chosen cut, what is the concrete emitted shape — + a `const`/`static` manifest table, a generated `impl ClassView`, or fn + stubs? How does it carry the GAP-CONST-OVERLOAD merged-method case honestly? +4. **container-architect:** does the emitted output respect the locked + NodeRow/ValueTenant/ValueSchema/ClassView layout — zero new variant, zero + stride change? Any hidden lock break in a generated `impl`? +5. **integration-lead:** placement (ruff vs tesseract-rs vs standalone) + + sequencing. What does manifest-first unblock; what stays gated on Option A / + byte-parity? Is step 2 the right next move, or does D (harvester polish: + GAP-CONST-OVERLOAD fix) come first so the manifest is collision-free? + +### Consolidation — 5-agent council (2026-06-17, step 2) + +**MAJOR REFRAME (3 of 5 independently): the plan's premise is false against the +code.** `ClassView` (`class_view.rs:152-249`) is a **field/render** vocabulary +(`fields() -> &[FieldRef]`, `template()`, `value_schema()`, `edge_codec_flavor()`) +— it has **no method-resolution surface**; the string `has_function` does not +appear anywhere in `lance-graph-contract`. So "emit a `classid → ClassView` +method manifest targeting the existing trait" targets a home that does not exist. + +| Agent | Verdict | Load-bearing finding | +|---|---|---| +| core-first-architect | **re-scope; none of the 3 cuts as written** | The 3 cuts are PARALLEL-MODEL (1, as framed) / RESIDUE-CORE (2) / out-of-scope (3). **Discovered `codegen_spine.rs` — a SHIPPED codegen contract the plan never cites:** `TripletProjection` + `roundtrip_eq` (build-time loss falsifier), `Genericity` (codegen-vs-runtime marker), `manifest.rs` (build.rs-generated `&'static` const data is already a Core-sanctioned pattern). Proposes "projection-first": emit via `TripletProjection` validated by the shipped `roundtrip_eq`, sourced from `ruff_spo_triplet::CppMethod` directly (no new struct), no `impl ClassView`, no classid mint. GAP-CONST-OVERLOAD then becomes an **observable round-trip failure**, not a documented-and-ignored merge. Rule: emitter may only produce `impl TripletProjection` const (validated by `roundtrip_eq`) OR `ocr.rs::to_node_row`-shape bodies; never a new Rust type for a C++ class/method; never an `impl ClassView` method member (that's unsanctioned EXTEND-CORE on locked canon). | +| core-gap-auditor | **EXTEND-CORE (minimal)** | A *method*-resolution manifest has no home on today's ClassView. Minimal additive delta: a `MethodRef` POD + a free `classid_methods(classid) -> &[MethodRef]` `LazyLock` registry — the **method-axis sibling of `classid_read_mode`/`BUILTIN_READ_MODES`** — never a string-mangled `FieldRef`, never an adapter that carries its own table. No node bytes / ValueTenant / stride / version bump. **Honest split:** a *value/field* manifest IS Core-ready today (the `ocr.rs` precedent); only the *method* manifest needs the delta. Manifest-first sidesteps Option A (content-store). GAP-CONST-OVERLOAD at manifest level = benign "one entry not two" iff identity key = `(name, param_types)` (which `reassemble` already uses). | +| adapter-shaper | **manifest-first, but NOT `impl ClassView`** | `FieldRef` is `String`-backed (`ontology.rs:467`) → a `const` ClassView field table is impossible. Thinnest TARGETS-CORE shape: a `classid → &'static [MethodSig]` registry mirroring `BUILTIN_READ_MODES`, with a NEW `&'static str`-backed `MethodSig`/`Receiver` type in a new `lance-graph-contract::codegen_manifest` module. Mapped `CppMethod`→`MethodSig` (name, ordered params, ret, receiver=Const/Static, overrides); dropped the body-shaping flags (pure_virtual/constexpr/noexcept/operator/requires). GAP-CONST-OVERLOAD → `merged_overload: true` flag + comment (machine-checkable). Frankenstein guard = hand-curated deny-list (`id_to_unichar_ext`, `CleanupString`, LSTM/ELIST) in `ruff_cpp_codegen/src/hand_port_denylist.rs`. PARITY markers in `//!` headers. | +| container-architect | **ADDITIVE-CONFIRMED (all 3 cuts)** | Generated code is ClassView-side; never touches NodeRow/ValueTenant/ValueSchema/stride/`ENVELOPE_LAYOUT_VERSION`. Single invariant: generated impls **SELECT** existing `ValueSchema`/`EdgeCodecFlavor` presets, never **CONSTRUCT** a layout — unbreakable because the enums have no byte-offset constructor. Adding a sibling `codegen_manifest` module is additive (same posture as `ocr.rs` adding `LayoutBlock` beside `NodeRow`). | +| integration-lead | **placement (a) + D-first** | Emitter = new `ruff_cpp_codegen` crate in ruff next to `ruff_spo_triplet`, **emit-text-only — compile NOTHING against lance-graph** (consumes `ModelGraph`, only NAMES contract types as text; a ruff→lance-graph-contract compile edge is FORBIDDEN — inverts the contract's consumers-depend-on-it arrow; verified zero `lance` in ruff manifests). **Run D (GAP-CONST-OVERLOAD harvester fix) BEFORE the emitter** so the first manifest is collision-free (0/67 not 19/67); D is small, in-ruff, operator-gated. Manifest-first = maximal in-env slice; bodies gated on Option A, parity on Option B. | + +### The unresolved fork (for the 3-brutal panel) + +All 5 agree: **manifest-first** (not stubs/bodies), **additive**, **placement = +`ruff_cpp_codegen` emit-text-only**, **D before the emitter**, **PARITY markers + +Frankenstein deny-list**. The genuine disagreement the brutal panel must settle: + +- **A — `MethodSig` registry (adapter-shaper + core-gap-auditor):** mint a minimal + `MethodSig` POD + `classid_methods` `LazyLock` registry (method-axis sibling of + `classid_read_mode`). Runtime-dispatchable; EXTEND-CORE but additive. +- **B — `TripletProjection` projection-first (core-first-architect):** ride the + SHIPPED `codegen_spine::TripletProjection`/`roundtrip_eq`; emit NO new type; + GAP-CONST-OVERLOAD becomes an observable `roundtrip_eq` failure. Warns that a + new `MethodSig` re-implements `ruff_spo_triplet::CppMethod` = the parallel-model + trap, and that re-emitting the harvest as a Rust table is a **tautology** + ("emit the manifest, test it matches the harvest it came from"). + +The brutal panel adjudicates: (1) is `MethodSig` a justified minimal EXTEND-CORE +or the parallel-model/`CppMethod`-duplication trap? (2) does the shipped +`codegen_spine::TripletProjection` actually fit a method manifest, or is that +stretching a triple-projection contract? (3) is manifest-first real codegen or a +tautology — and does D go first? From b17b69d6a110e5ec6c99f994e70e6df0988cdcd6 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 15:13:44 +0000 Subject: [PATCH 4/9] =?UTF-8?q?plans:=20C=20step=202=20=E2=80=94=203-bruta?= =?UTF-8?q?l=20panel=20+=20final=208/8=20decision=20(D-first,=20cv-aware?= =?UTF-8?q?=20IRI)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The brutal panel resolved the A-vs-B fork as a false binary (honest-tester): B's codegen_spine::roundtrip_eq pattern is the build-time GATE, A's MethodSig shape is the emitted-text target, A's runtime registry is a deferred additive EXTEND-CORE. baton-auditor's decisive correction (confirmed by honest-tester): the const-overload merge is UPSTREAM in expand's (s,p,o) dedup, so the round-trip cannot observe it — it is a fixed point. Therefore D (cv-aware method IRI) must run FIRST, unanimously (behavior-monitor's emitter-first rationale refuted). Final order: D (cv-aware IRI, autonomous correctness fix — adds no predicate, falsifier CPP-REASSEMBLE-RT 48/67 -> 67/67) -> emitter scaffold (ruff_cpp_codegen, emit-text-only, ruff-side round-trip gate, no classid mint) -> MethodSig EXTEND-CORE in lance-graph (additive) -> wire/byte-parity (operator). Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 41 +++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index 4496c73a..b330d508 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -307,3 +307,44 @@ or the parallel-model/`CppMethod`-duplication trap? (2) does the shipped `codegen_spine::TripletProjection` actually fit a method manifest, or is that stretching a triple-projection contract? (3) is manifest-first real codegen or a tautology — and does D go first? + +### 3-brutal panel (2026-06-17, step 2) + +| Critic | Verdict | Load-bearing finding | +|---|---|---| +| adk-behavior-monitor | A=tautology, B=falsifier, **emitter-before-D** | Option A's "manifest matches harvest" test is a tautology (AP2) + duplicates `CppMethod` (AP3). `roundtrip_eq` checks `decompile(project(input)) == input` *internally* (verified `codegen_spine.rs:138-183`), so it's a real falsifier. Claimed GAP-CONST-OVERLOAD becomes an observable round-trip failure and proposed building against the dirty corpus first as the fixture. **[Refuted on the const-overload point — see below.]** | +| baton-handoff-auditor | **A (re-scoped), D-first, no classid mint** | **Decisive correction:** the const-overload merge happens UPSTREAM in `expand`'s `(s,p,o)` dedup (`expand.rs:123-128`, `:581-585`) — both overloads collapse to one IRI *before* any projection/reassembly. So `roundtrip_eq` sees the collapsed set and passes; it CANNOT observe the gap. `MethodSig`≠`CppMethod`-dup (harvest-IR vs `&'static` Core-registry, the `ReadMode`/`BUILTIN_READ_MODES` posture; verified neither exists yet). Emitter must emit **name/IRI-keyed text and NEVER mint a classid** (the `ocr.rs::to_node_row(classid,…)` precedent takes classid as a caller param). Forbidden ruff→lance edge confirmed absent. | +| brutally-honest-tester | **HOLD → build with 3 fixes; A-vs-B is a FALSE BINARY** | `codegen_spine::TripletProjection` genuinely fits (generic over `type Const`; a method manifest IS a valid `Const`) — but core-first-architect's "emit NO new type" is self-contradictory (the trait REQUIRES a `Const` type). The synthesis: **B's `roundtrip_eq` machinery = the build-time GATE; A's `MethodSig` shape = the emitted text target; A's runtime registry = DEFERRED canon-review.** Confirmed const-overload is a round-trip fixed point (invisible) → carry `merged_overload` explicitly AND **D-first** (more firmly than integration-lead). The one discipline that separates codegen from tautology: gate the manifest `Const` with a round-trip against the **LIVE** harvested triples, never a hand-authored table. | + +### FINAL DECISION (8/8 consolidated) + +**Build the emitter — but D first, with B's round-trip as the gate and A's shape +as the output.** Execution order: + +1. **D — cv-aware method IRI (NOW, autonomous correctness fix).** The current + IRI scheme silently merges 19/67 const/non-const overloads — a **lossy-harvest + defect** the falsifier quantified, not a feature. Append ` const` to the method + IRI when `is_const` (in `expand`), reconstruct it in `reassemble`, and make the + override target cv-aware in `clang_walker`. **Reclassified from "operator-gated" + to autonomous correctness fix:** it adds NO predicate (vocab stays 54), it is + ruff-internal, and it has a hard falsifier — `CPP-REASSEMBLE-RT` must go + **48/67 → 67/67** byte-exact. (Reversible: a string format in three functions.) +2. **Emitter scaffold (`ruff_cpp_codegen`, emit-text-only).** Walks the + reassembled `ModelGraph`, emits `MethodSig`-shaped Rust **text** naming + `lance_graph_contract` types; `ruff_spo_triplet`-only dep (no lance edge). + **Gate = a ruff-side round-trip** (the `TripletProjection` PATTERN over + `ruff_spo_triplet::Triple`: emit manifest `Const` → decompile → assert + `(s,p,o)`-set-equal to the live harvested triples). NEVER mints a classid + (name/IRI-keyed). PARITY markers + hand-curated Frankenstein deny-list. +3. **`MethodSig` EXTEND-CORE in lance-graph (additive, council-reviewed).** A + `&'static str`-backed POD + `classid_methods` `LazyLock` registry — the + method-axis sibling of `classid_read_mode`. container-architect: + ADDITIVE-CONFIRMED (generated impls SELECT presets, never CONSTRUCT layout). + Lands when the emitter's output is wired; classid binding stays OGAR-side. +4. **Wire + byte-parity (Option B, operator's leptonica host)** — the only path + to CONJECTURE→FINDING; unchanged. + +**Corrections folded in:** the gate is a live-triple round-trip (not a +self-golden); GAP-CONST-OVERLOAD is fixed at the source by D (not relied on as a +round-trip failure); A and B are lifecycle stages, not a choice. Sequence: +**D → emitter scaffold → MethodSig EXTEND-CORE → wire/parity.** From c967c1438e827e8797f3140136667d4f037cc230 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 15:32:42 +0000 Subject: [PATCH 5/9] =?UTF-8?q?plans:=20D=20LANDED=20=E2=80=94=20CPP-REASS?= =?UTF-8?q?EMBLE-RT=2067/67;=20falsifier=20corrected=20the=20const=20assum?= =?UTF-8?q?ption?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Records D's outcome: the cv-aware method IRI + two round-trip-metric fixes (cpp_projection dedup; is_const in the methods sort key) closed the entire collision tail (48/67 -> 67/67, now a hard gate). The falsifier overturned the council's "19/67 = const overloads" inference: only 3 were const; 13 were benign duplicate template_instantiates, 2 duplicate-harvested methods, 1 a sort-order artifact. GAP-CONST-OVERLOAD resolved. Next: emitter scaffold (ruff_cpp_codegen) then the MethodSig EXTEND-CORE. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index b330d508..8fb10d7b 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -348,3 +348,29 @@ as the output.** Execution order: self-golden); GAP-CONST-OVERLOAD is fixed at the source by D (not relied on as a round-trip failure); A and B are lifecycle stages, not a choice. Sequence: **D → emitter scaffold → MethodSig EXTEND-CORE → wire/parity.** + +### D — LANDED (2026-06-17, AdaWorldAPI/ruff) + +cv-aware method IRI shipped: `expand` appends ` const` when `is_const`, +`clang_walker` does the same for the override target, `reassemble` reconstructs +it. **`CPP-REASSEMBLE-RT` went 48/67 → 67/67 byte-exact, now a hard gate.** + +**The falsifier corrected the council's own assumption.** "19/67 = const +overloads" was an inference; the cv-aware IRI fixed only **3**. Tracing the rest +(per-class methods/templates/fields delta) showed the council had conflated three +distinct causes: **13** were benign duplicate `template_instantiates` that +`expand` dedups but `cpp_projection` did not; **2** were duplicate-harvested +methods (same cause); **1** (UnicityTable) was an equal-`(name,params)`-sort-key +ordering artifact for a const/non-const pair. Two round-trip-metric fixes +followed — `cpp_projection` now de-duplicates every collection (mirroring +`expand`'s `(s,p,o)` dedup), and the methods sort key includes `is_const`. Net: +the round-trip is now a faithful, total-order, deduped projection; real +collisions (same key, different content) still surface, exact duplicates collapse. + +This is the C-FIRST doctrine working as intended: build the cheap in-env +falsifier, run it on real data, and let it overturn the synthesis — the council +*assumed* a single const-overload cause; the measurement found three, and only +one needed D. `GAP-CONST-OVERLOAD` is RESOLVED; no merged-adapter known-gap +remains. **Next: the emitter scaffold (`ruff_cpp_codegen`, autonomous, +emit-text + ruff-side round-trip gate); then the `MethodSig` EXTEND-CORE in +lance-graph (additive, the emitted text's compile target).** From b5041595a9959b6c6e042f581d5482e6d5f1e259 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 18:30:47 +0000 Subject: [PATCH 6/9] =?UTF-8?q?plans:=20emitter=20scaffold=20LANDED=20?= =?UTF-8?q?=E2=80=94=20ruff=5Fcpp=5Fcodegen,=20CPP-CODEGEN-RT=20on=20real?= =?UTF-8?q?=20corpus?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The C-FIRST step-2 emitter shipped in ruff: project(ModelGraph)->MethodSig manifest + render to lance-graph-naming Rust text (emit-text-only, no lance-graph edge), gated by a decompile==expand signature-plane round-trip with teeth (dropped-method test proves it fails). CPP-CODEGEN-RT on real ccutil: 67 classes, 857 methods -> 124 KB MethodSig manifest, round-trip holds, PARITY markers + Frankenstein deny-list present. Next (operator-gated, additive canon growth): the MethodSig EXTEND-CORE in lance-graph-contract. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .../plans/transcode-extend-core-probe-v1.md | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index 8fb10d7b..628f7caa 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -374,3 +374,32 @@ one needed D. `GAP-CONST-OVERLOAD` is RESOLVED; no merged-adapter known-gap remains. **Next: the emitter scaffold (`ruff_cpp_codegen`, autonomous, emit-text + ruff-side round-trip gate); then the `MethodSig` EXTEND-CORE in lance-graph (additive, the emitted text's compile target).** + +### Emitter scaffold — LANDED (2026-06-17, AdaWorldAPI/ruff) + +`ruff_cpp_codegen` shipped (depends on `ruff_spo_triplet` only — the forbidden +`ruff → lance-graph` edge confirmed absent): + +- **`project(&ModelGraph) → Vec`** extracts the method plane + (`MethodSig` = name, ordered params, return, `is_const`, `is_static`, override; + body-shaping flags dropped). **`render(&[ClassManifest]) → String`** emits + Rust **text** naming `lance_graph_contract::codegen_manifest::MethodSig` — + emit-text-only, deterministic, escaped (comma-bearing templated params stay one + quoted element), with `PARITY: UNRUN` headers and the hand-curated Frankenstein + deny-list (`id_to_unichar_ext`, `CleanupString` → `// HAND-PORT`). +- **The gate has teeth (not a self-golden):** `decompile(project(g))` must equal + `expand(g)` on the signature plane — the `codegen_spine::roundtrip_eq` pattern + over the *live* triples, implemented over `ruff_spo_triplet::Triple` to stay + lance-graph-free. `round_trip_detects_a_dropped_method` proves it can fail. +- **`CPP-CODEGEN-RT` (gated e2e on the real corpus):** ccutil harvest → 67 + classes, **857 methods → a 124 KB `MethodSig` manifest**; the signature-plane + round-trip holds and the render carries PARITY + one literal per method. + classid minting stays OGAR-side (name-keyed text). 10 + 16 tests green. + +**Next (operator-gated): the `MethodSig` EXTEND-CORE in `lance-graph-contract`** +— a `&'static str`-backed `MethodSig` POD + `codegen_manifest` module (additive, +container-architect ADDITIVE-CONFIRMED), the compile target the emitted text +names. It touches operator-locked canon (additively), so unlike D and the +scaffold (ruff-internal correctness/codegen) it is the deliberate-Core-growth +step the doctrine says to file + review before landing. Then: wire the generated +crate in tesseract-rs and run byte-parity (Option B, operator's leptonica host). From 7441cd383887b3ffda90f0d3de7ff55441a59c84 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 18:49:52 +0000 Subject: [PATCH 7/9] =?UTF-8?q?lance-graph-contract:=20codegen=5Fmanifest:?= =?UTF-8?q?:MethodSig=20=E2=80=94=20the=20C++=20codegen=20compile=20target?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The MethodSig EXTEND-CORE (C-FIRST step 2's deferred-runtime-registry piece). A new codegen_manifest module: - MethodSig: the dispatch-relevant C++ method signature in a const-constructible shape (all fields &'static: name, params: &'static [&'static str], ret, is_const, is_static, overrides). It is the exact literal ruff_cpp_codegen::render emits, so the generated text now has a real compile target. The &'static shape is load-bearing: class_view::FieldRef is String-backed and cannot appear in a const; MethodSig is the method-axis sibling that can. - ClassMethods{classid, methods} + methods_for(registry, classid): the registry-entry type + pure zero-fallback lookup (unregistered classid -> empty slice). classid is bound OGAR-side, never minted here; the runtime classid->methods registry DATA is generated downstream (consumer repo), NOT stored here (honest-tester's "defer the runtime registry"). Additive (container-architect ADDITIVE-CONFIRMED): a sibling module, zero NodeRow/ValueTenant/ValueSchema/stride/ENVELOPE_LAYOUT_VERSION impact. Body-shaping flags (pure-virtual/constexpr/noexcept/operator/requires) are out of scope. Board hygiene: LATEST_STATE Contract Inventory updated same commit (D-CPP-CODEGEN-1). +2 tests (const-constructibility proof + zero-fallback); 640 contract lib green; clippy -D warnings clean. C-FIRST: D + emitter scaffold + MethodSig EXTEND-CORE all landed; the in-env arc is complete. Remaining is operator-gated (tesseract-rs wiring + byte-parity). Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .claude/board/LATEST_STATE.md | 2 + .../plans/transcode-extend-core-probe-v1.md | 32 ++++ .../src/codegen_manifest.rs | 144 ++++++++++++++++++ crates/lance-graph-contract/src/lib.rs | 1 + 4 files changed, 179 insertions(+) create mode 100644 crates/lance-graph-contract/src/codegen_manifest.rs diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index df780a8e..4a6fd2ff 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -82,6 +82,8 @@ ## Current Contract Inventory (lance-graph-contract) +> **2026-06-17 — ADDED (D-CPP-CODEGEN-1, C-FIRST step 2 compile target)**: `lance_graph_contract::codegen_manifest::{MethodSig, ClassMethods, methods_for}` — the Core-side target of the C++ method-resolution manifest emitted by `ruff_cpp_codegen` (the Tesseract AST-DLL pipeline's stage 2). `MethodSig` is the dispatch-relevant signature in a **`const`-constructible** shape (all fields `&'static`: `name`, `params: &'static [&'static str]`, `ret`, `is_const`, `is_static`, `overrides`) — the method-axis sibling of `class_view::ClassView`'s field projection, deliberately NOT `String`-backed (a generated `const X: &[MethodSig] = &[MethodSig { .. }]` must compile; `FieldRef` is `String`-backed and cannot). `ClassMethods{classid, methods}` is the registry ENTRY the generated code emits (classid bound OGAR-side, never minted here); `methods_for(registry, classid) -> &'static [MethodSig]` is the pure lookup with zero-fallback (unregistered classid → empty slice). **Additive** (container-architect ADDITIVE-CONFIRMED): a sibling module, zero `NodeRow`/`ValueTenant`/`ValueSchema`/stride/`ENVELOPE_LAYOUT_VERSION` impact; the runtime `classid→methods` registry DATA lives downstream (generated in the consumer repo), not here. Body-shaping flags (pure-virtual/constexpr/noexcept/operator/requires) are out of scope (they drive body generation, not the signature manifest). The 8-agent step-2 council's deferred-runtime-registry resolution. +2 tests (const-constructibility proof + zero-fallback lookup); 640 contract lib green; clippy `-D warnings` clean. Plan: `.claude/plans/transcode-extend-core-probe-v1.md` (C step 2). Consumer: `ruff_cpp_codegen::render` (AdaWorldAPI/ruff) names this type in emit-text-only output. + > **2026-06-16 — ADDED (4-task unblock-cascade)**: `lance_graph_contract::hhtl::NiblePath::{from_guid_prefix(&NodeGuid) -> Option, prefix(depth: u8) -> Option}` — the ontology-side keystone follow-up of #498's `classid → ReadMode` LE contract. The 20-nibble `classid · HEEL · HIP · TWIG` prefix is deterministically folded to 16 (the canon-reserved high `u16` of classid drops); returns `None` when the fold would be lossy (callers don't get silent collisions). `prefix(d)` is the O(1) single-shot ancestor view that satisfies `prefix(d).is_ancestor_of(self)` for every `d ≤ self.depth` — the routing-cache view of a deeper class path. **One layer up** in `cognitive-shader-driver::MailboxSoA`: `impl MailboxSoaView + MailboxSoaOwner` (cherry-pick of `jolly-cori-clnf9::463d71b`) + the `pub phase: KanbanColumn` field — the in-RAM Rubicon owner the contract's `MailboxSoaOwner` had no real implementor for (integrated-cognitive-planner-v1 §2 Seam #3 closed). In `lance_graph::graph::scheduler`: `LanceVersionScheduler` — D-MBX-9-IN core impl over `VersionedGraph::versions()`, generic over the inner `VersionScheduler` policy (closes `E-SUBSTRATE-IS-THE-SCHEDULER`'s OUT-direction). In `surreal_container::view`: `SurrealMailboxView<'a>` + `read_via_kv_lance()` (D-PG-6 contract slice) — the SurrealQL read-glove the integrator wires once the cold-build of the surrealdb fork is taken; the contract surface is available today. Plus `SurrealContainerError::BlockedColdBuild` — typed signal for callers to pattern-match the cold-build gate (distinct from the pre-existing `Blocked` variant which signals coordinate/API gaps). Zero-dep contract additions (+7 hhtl tests, 632 lib green); cognitive-shader-driver +1 driving-loop test (86 lib green); lance-graph::scheduler new module (+5 tests, real tempdir Lance); surreal_container::view new module (+4 tests). All four green; clippy `-D warnings` clean on the new files. EPIPHANIES `E-UNBLOCK-CASCADE-1` records the convergence of three independent landings onto the single `MailboxSoaView` trait surface. > **2026-06-09 — ADDED (D-IDENTITY-1, Phase A of identity-architecture)**: `lance_graph_contract::identity::{NodeGuid([u8;16]), IDENTITY_LAYOUT_VERSION}` — the workspace's first **stable binary instance identity**: a structured 128-bit UUIDv8 (RFC 9562) = the HHTL nibble-address **formalized + namespaced**. **Composed from existing committed scalars, never re-invented** (Agent A sweep confirmed the 128-bit id space was empty): octets carry `namespace:u8 | entity_type:u16 | kind:u8` (the `SchemaPtr.packed` convention) ⊕ a truncated `NiblePath` routing prefix (`PREFIX_NIBBLES=4`) ⊕ a 22-bit `shape_hash` (truncated `StructuralSignature`) ⊕ a 24-bit `local`, with UUIDv8 version(=8)/variant(=0b10) at their RFC-fixed positions + an `IDENTITY_LAYOUT_VERSION` stamp. **Eineindeutigkeit**: `entity_type` is the canonical exact class identity; the `NiblePath` prefix is the bijective DERIVED view (a *truncated* prefix can't be the identity — deep classes collide past it; the prefix `is_ancestor_of` the full path). Five readings: resolve (`entity_type`) / route (`niblepath`) / witness (frozen bytes + merkle) / ground-truth (`shape_hash` drift) / dispatch-to-store (`as_bytes` → `EntityKey`). Also added `hhtl::NiblePath::from_packed` (inverse of `packed`). Zero-dep; 599 contract lib tests (+15: field-isolation matrix, UUIDv8 gates, ancestor-prefix invariant, Display=canonical-UUID); clippy `-D warnings` clean; fmt clean. Plans: `identity-architecture-exists-vs-needs-v1.md` (exists-vs-needs map + phases A→H), `cognitive-write-roundtrip-substrate-v1.md`. Epiphany: E-IDENTITY-WHITEBOX-1. diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index 628f7caa..f503774b 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -403,3 +403,35 @@ names. It touches operator-locked canon (additively), so unlike D and the scaffold (ruff-internal correctness/codegen) it is the deliberate-Core-growth step the doctrine says to file + review before landing. Then: wire the generated crate in tesseract-rs and run byte-parity (Option B, operator's leptonica host). + +### MethodSig EXTEND-CORE — LANDED (2026-06-17, AdaWorldAPI/lance-graph) + +`lance_graph_contract::codegen_manifest` shipped (the council pre-review = +file + review; container-architect ADDITIVE-CONFIRMED): + +- **`MethodSig`** — the dispatch signature in a `const`-constructible shape (all + fields `&'static`: `name`, `params: &'static [&'static str]`, `ret`, + `is_const`, `is_static`, `overrides`). This is the exact literal + `ruff_cpp_codegen::render` emits, so the generated text now has a real compile + target. The `&'static` shape is the whole point — `class_view::FieldRef` is + `String`-backed and cannot appear in a `const`; `MethodSig` is the method-axis + sibling that can. +- **`ClassMethods{classid, methods}`** + **`methods_for(registry, classid)`** — + the registry-entry type + pure zero-fallback lookup. classid is bound + OGAR-side (never minted here); the runtime `classid→methods` registry DATA is + generated downstream (consumer repo), NOT stored in lance-graph — the + honest-tester's "defer the runtime registry" honored. +- **Additive**: a sibling module, zero `NodeRow`/`ValueTenant`/`ValueSchema`/ + stride/`ENVELOPE_LAYOUT_VERSION` impact. Board hygiene: LATEST_STATE Contract + Inventory updated in the same commit (D-CPP-CODEGEN-1). +2 tests + (const-constructibility proof — the load-bearing property — + zero-fallback + lookup); 640 contract lib green; clippy `-D warnings` clean. + +**C-FIRST status:** D ✓ → emitter scaffold ✓ → MethodSig EXTEND-CORE ✓. The +in-env, lance-graph-internal arc of the pipeline is complete: a C++ harvest now +flows harvest → reassemble → project → render → a `MethodSig` manifest whose +type exists in the Core. **Remaining (operator-gated, out of this env):** wire +the generated crate into tesseract-rs (needs the leptonica build env) and run +`PROBE-OGAR-ADAPTER-UNICHARSET` byte-parity (Option B) — the only path to +CONJECTURE→FINDING. Everything to here is CONJECTURE per the doctrine; the +`PARITY: UNRUN` markers on every generated file say so. diff --git a/crates/lance-graph-contract/src/codegen_manifest.rs b/crates/lance-graph-contract/src/codegen_manifest.rs new file mode 100644 index 00000000..1f79ae85 --- /dev/null +++ b/crates/lance-graph-contract/src/codegen_manifest.rs @@ -0,0 +1,144 @@ +//! Codegen manifest types — the Core-side target of the C++ method-resolution +//! manifest emitted by `ruff_cpp_codegen` (the AST-DLL pipeline's stage 2). +//! +//! `ruff_cpp_spo` harvests a C++ corpus into SPO triples; `ruff_spo_triplet` +//! reassembles the method plane; `ruff_cpp_codegen` renders it as Rust source +//! that names [`MethodSig`]. This module is the **compile target** of that +//! generated text. +//! +//! # Why `&'static`, not `String` +//! +//! Every field is `&'static` so a generated +//! `const X: &[MethodSig] = &[MethodSig { .. }]` compiles. The render-side +//! manifest could not target [`crate::class_view::ClassView`]'s `fields()` +//! because `FieldRef` is `String`-backed (not `const`-constructible); +//! `MethodSig` is the method-axis sibling, deliberately all-`&'static`. +//! +//! # What lives here vs. downstream +//! +//! This module provides the **type + lookup**. The **data** — the per-class +//! `const … : &[MethodSig]` tables and the `&[ClassMethods]` registry — is +//! generated downstream (in the consumer repo, e.g. tesseract-rs) and is NOT +//! held here; lance-graph mints no C++ classids and stores no method tables. +//! The classid is bound OGAR-side (the `ocr.rs::to_node_row(classid, …)` +//! precedent — classid is a parameter, never minted by the manifest). A runtime +//! `classid → methods` registry is intentionally deferred: this is the minimal +//! additive Core growth (`MethodSig` is the dispatch signature; the body +//! adapters and a populated registry come later). +//! +//! `MethodSig` carries only the dispatch-relevant signature (name, ordered +//! params, return, `is_const`/`is_static`, override target). The body-shaping +//! flags the harvest also captures (`is_pure_virtual` / `constexpr` / +//! `noexcept` / operator-kind / `requires`) drive body generation, not the +//! signature manifest, and are not represented here. + +/// One C++ method's dispatch-relevant signature, in a `const`-constructible +/// shape (all fields `&'static`). This is the exact literal +/// `ruff_cpp_codegen::render` emits. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct MethodSig { + /// Bare method name (e.g. `unichar_to_id`, `operator==`). + pub name: &'static str, + /// Parameter types in signature order, verbatim + /// (e.g. `&["const char *", "int"]`). + pub params: &'static [&'static str], + /// Return type, verbatim. `None` for void / constructors / destructors. + pub ret: Option<&'static str>, + /// `T method() const;` — a const-qualified (read-accessor) member. + pub is_const: bool, + /// `static T method();` — a class-level member (no implicit `this`). + pub is_static: bool, + /// The fully-qualified overridden base method (cv-aware), if any. + pub overrides: Option<&'static str>, +} + +impl MethodSig { + /// Number of parameters. + #[must_use] + pub const fn arity(&self) -> usize { + self.params.len() + } + + /// Whether this method overrides a virtual base method. + #[must_use] + pub const fn is_override(&self) -> bool { + self.overrides.is_some() + } +} + +/// One class's method manifest, keyed by classid — the registry ENTRY the +/// generated code emits. The classid is bound OGAR-side; this type only +/// associates an already-minted classid with its `const` method table. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ClassMethods { + /// The OGAR classid this manifest belongs to. + pub classid: u32, + /// The class's methods (a generated `const` table). + pub methods: &'static [MethodSig], +} + +/// Resolve a classid to its method manifest within a generated `registry` +/// slice. The Core provides the lookup; the `registry` data is generated +/// downstream. Returns an empty slice for an unregistered classid (the +/// zero-fallback ladder: an unknown class resolves to "no methods", never a +/// panic). +#[must_use] +pub fn methods_for(registry: &[ClassMethods], classid: u32) -> &'static [MethodSig] { + registry + .iter() + .find(|entry| entry.classid == classid) + .map_or(&[], |entry| entry.methods) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The load-bearing property: `MethodSig` is `const`-constructible. This is + /// the exact shape `ruff_cpp_codegen::render` emits — if it ever stopped + /// compiling in a `const`, the generated manifests would too. (Contrast + /// `FieldRef`, which is `String`-backed and cannot appear in a `const`.) + const SAMPLE: &[MethodSig] = &[ + MethodSig { + name: "unichar_to_id", + params: &["const char *"], + ret: Some("UNICHAR_ID"), + is_const: true, + is_static: false, + overrides: None, + }, + MethodSig { + name: "kSpaceId", + params: &[], + ret: Some("UNICHAR_ID"), + is_const: false, + is_static: true, + overrides: None, + }, + ]; + + const REGISTRY: &[ClassMethods] = &[ClassMethods { + classid: 0x0001, + methods: SAMPLE, + }]; + + #[test] + fn const_manifest_constructs_and_reads() { + assert_eq!(SAMPLE.len(), 2); + assert_eq!(SAMPLE[0].name, "unichar_to_id"); + assert_eq!(SAMPLE[0].arity(), 1); + assert!(SAMPLE[0].is_const); + assert!(!SAMPLE[0].is_override()); + assert!(SAMPLE[1].is_static); + assert_eq!(SAMPLE[1].arity(), 0); + } + + #[test] + fn methods_for_resolves_classid_with_zero_fallback() { + assert_eq!(methods_for(REGISTRY, 0x0001).len(), 2); + assert!( + methods_for(REGISTRY, 0xDEAD).is_empty(), + "an unregistered classid resolves to no methods (zero-fallback)" + ); + } +} diff --git a/crates/lance-graph-contract/src/lib.rs b/crates/lance-graph-contract/src/lib.rs index 2f2f7739..b1164585 100644 --- a/crates/lance-graph-contract/src/lib.rs +++ b/crates/lance-graph-contract/src/lib.rs @@ -44,6 +44,7 @@ pub mod callcenter; pub mod cam; pub mod canonical_node; pub mod class_view; +pub mod codegen_manifest; pub mod codegen_spine; pub mod cognitive_shader; pub mod collapse_gate; From c52b0d62462a3c5cb11e0faede040cd937bc98dd Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 19:31:32 +0000 Subject: [PATCH 8/9] =?UTF-8?q?lance-graph-contract:=20UniCharSet=20conten?= =?UTF-8?q?t=20store=20=E2=80=94=20the=20byte-parity=20probe's=20Rust=20si?= =?UTF-8?q?de?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The deferred Option A content-store tier, built (operator's "keep building here" + the leptonica-is-an-install-not-a-transcode epiphany). New unicharset module: UniCharSet (deepnsm::Vocabulary-shaped: reverse id->unichar + lookup unichar->id), load_from_str/load_from_file parsing the .unicharset text format (line 1 = count; first whitespace token per line = unichar; id = position; property columns ignored — the old_style_included_ plain-table scope), id_to_unichar/unichar_to_id (the two adapter leaves), and dump() rendering the \t table matching the C++ oracle. This is the Rust side of PROBE-OGAR-ADAPTER-UNICHARSET. The unicharset path is pure text parsing — ZERO leptonica (never touches Pix) — so it builds and unit-tests in-env with no C deps. leptonica is only an *install* (a link dep of the C++ oracle harness), never a transcode and never in the Rust path. Byte parity is now one `diff`: combine_tessdata to get eng.unicharset, a ~10-line libtesseract harness dumps id_to_unichar, `cargo run --example unicharset_dump` dumps the Rust side, diff. Byte-identical => CONJECTURE -> FINDING. Additive (sibling content-store module, zero NodeRow/tenant impact). Board: LATEST_STATE Contract Inventory (D-UNICHARSET-1). +4 tests + the unicharset_dump example; 644 contract lib green; clippy -D warnings + fmt clean. The classid->&UniCharSet LazyLock resolver (OGAR wiring) remains the follow-up. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .claude/board/LATEST_STATE.md | 2 + .../plans/transcode-extend-core-probe-v1.md | 23 ++ .../examples/unicharset_dump.rs | 39 ++++ crates/lance-graph-contract/src/lib.rs | 1 + crates/lance-graph-contract/src/unicharset.rs | 219 ++++++++++++++++++ 5 files changed, 284 insertions(+) create mode 100644 crates/lance-graph-contract/examples/unicharset_dump.rs create mode 100644 crates/lance-graph-contract/src/unicharset.rs diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 4a6fd2ff..89313161 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -82,6 +82,8 @@ ## Current Contract Inventory (lance-graph-contract) +> **2026-06-17 — ADDED (D-UNICHARSET-1, byte-parity probe Rust side)**: `lance_graph_contract::unicharset::{UniCharSet, UniCharSetError}` — the Tesseract `UNICHARSET` content-store tier (the Core-First doctrine's variable-length classid-keyed registry, `deepnsm::Vocabulary`-shaped: `reverse: Vec` id→unichar + `lookup: HashMap` unichar→id). `load_from_str`/`load_from_file` parse the `.unicharset` text format (line 1 = count, then the first whitespace token per line = unichar, id = position; property columns ignored — the `old_style_included_` plain-table scope); `id_to_unichar`/`unichar_to_id` are the two adapter leaves; `dump()` renders the `\t` table matching the C++ oracle. **The Rust side of `PROBE-OGAR-ADAPTER-UNICHARSET`** — pure text parsing, ZERO leptonica (the unicharset path never touches `Pix`), so it builds + unit-tests in-env; byte-parity is one `diff` against a libtesseract oracle harness on a leptonica-installed box (steps in `examples/unicharset_dump.rs`). Additive (a sibling content-store module, zero `NodeRow`/tenant impact). +4 tests (format parse, bijection round-trip, oracle-shape dump, typed errors) + the `unicharset_dump` example; 644 contract lib green; clippy `-D warnings` clean. Plan: `transcode-extend-core-probe-v1.md` (the deferred Option A content-store tier, now built for the probe). The classid→`&UniCharSet` `LazyLock` resolver remains the wiring follow-up. + > **2026-06-17 — ADDED (D-CPP-CODEGEN-1, C-FIRST step 2 compile target)**: `lance_graph_contract::codegen_manifest::{MethodSig, ClassMethods, methods_for}` — the Core-side target of the C++ method-resolution manifest emitted by `ruff_cpp_codegen` (the Tesseract AST-DLL pipeline's stage 2). `MethodSig` is the dispatch-relevant signature in a **`const`-constructible** shape (all fields `&'static`: `name`, `params: &'static [&'static str]`, `ret`, `is_const`, `is_static`, `overrides`) — the method-axis sibling of `class_view::ClassView`'s field projection, deliberately NOT `String`-backed (a generated `const X: &[MethodSig] = &[MethodSig { .. }]` must compile; `FieldRef` is `String`-backed and cannot). `ClassMethods{classid, methods}` is the registry ENTRY the generated code emits (classid bound OGAR-side, never minted here); `methods_for(registry, classid) -> &'static [MethodSig]` is the pure lookup with zero-fallback (unregistered classid → empty slice). **Additive** (container-architect ADDITIVE-CONFIRMED): a sibling module, zero `NodeRow`/`ValueTenant`/`ValueSchema`/stride/`ENVELOPE_LAYOUT_VERSION` impact; the runtime `classid→methods` registry DATA lives downstream (generated in the consumer repo), not here. Body-shaping flags (pure-virtual/constexpr/noexcept/operator/requires) are out of scope (they drive body generation, not the signature manifest). The 8-agent step-2 council's deferred-runtime-registry resolution. +2 tests (const-constructibility proof + zero-fallback lookup); 640 contract lib green; clippy `-D warnings` clean. Plan: `.claude/plans/transcode-extend-core-probe-v1.md` (C step 2). Consumer: `ruff_cpp_codegen::render` (AdaWorldAPI/ruff) names this type in emit-text-only output. > **2026-06-16 — ADDED (4-task unblock-cascade)**: `lance_graph_contract::hhtl::NiblePath::{from_guid_prefix(&NodeGuid) -> Option, prefix(depth: u8) -> Option}` — the ontology-side keystone follow-up of #498's `classid → ReadMode` LE contract. The 20-nibble `classid · HEEL · HIP · TWIG` prefix is deterministically folded to 16 (the canon-reserved high `u16` of classid drops); returns `None` when the fold would be lossy (callers don't get silent collisions). `prefix(d)` is the O(1) single-shot ancestor view that satisfies `prefix(d).is_ancestor_of(self)` for every `d ≤ self.depth` — the routing-cache view of a deeper class path. **One layer up** in `cognitive-shader-driver::MailboxSoA`: `impl MailboxSoaView + MailboxSoaOwner` (cherry-pick of `jolly-cori-clnf9::463d71b`) + the `pub phase: KanbanColumn` field — the in-RAM Rubicon owner the contract's `MailboxSoaOwner` had no real implementor for (integrated-cognitive-planner-v1 §2 Seam #3 closed). In `lance_graph::graph::scheduler`: `LanceVersionScheduler` — D-MBX-9-IN core impl over `VersionedGraph::versions()`, generic over the inner `VersionScheduler` policy (closes `E-SUBSTRATE-IS-THE-SCHEDULER`'s OUT-direction). In `surreal_container::view`: `SurrealMailboxView<'a>` + `read_via_kv_lance()` (D-PG-6 contract slice) — the SurrealQL read-glove the integrator wires once the cold-build of the surrealdb fork is taken; the contract surface is available today. Plus `SurrealContainerError::BlockedColdBuild` — typed signal for callers to pattern-match the cold-build gate (distinct from the pre-existing `Blocked` variant which signals coordinate/API gaps). Zero-dep contract additions (+7 hhtl tests, 632 lib green); cognitive-shader-driver +1 driving-loop test (86 lib green); lance-graph::scheduler new module (+5 tests, real tempdir Lance); surreal_container::view new module (+4 tests). All four green; clippy `-D warnings` clean on the new files. EPIPHANIES `E-UNBLOCK-CASCADE-1` records the convergence of three independent landings onto the single `MailboxSoaView` trait surface. diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index f503774b..e0730283 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -435,3 +435,26 @@ the generated crate into tesseract-rs (needs the leptonica build env) and run `PROBE-OGAR-ADAPTER-UNICHARSET` byte-parity (Option B) — the only path to CONJECTURE→FINDING. Everything to here is CONJECTURE per the doctrine; the `PARITY: UNRUN` markers on every generated file say so. + +### Unicharset content store + byte-parity probe — Rust side READY (2026-06-17) + +The deferred Option A content-store tier, built (the operator chose "keep +building here" + the leptonica-is-an-install-not-a-transcode epiphany): +`lance_graph_contract::unicharset::UniCharSet` — `deepnsm::Vocabulary`-shaped +(`reverse`/`lookup`), `load_from_str`/`load_from_file` + `id_to_unichar` / +`unichar_to_id` + `dump()`. **Pure text parsing, ZERO leptonica** (the unicharset +path never touches `Pix`), so it builds + unit-tests in-env (4 tests; 644 lib +green). The `unicharset_dump` example renders the oracle-shape table. + +**The byte-parity probe is now one diff, not a build.** leptonica is an *install* +(`apt-get libtesseract-dev libleptonica-dev`), never a transcode — it's only a +*link* dep of the C++ oracle harness, never in the Rust path. So +`PROBE-OGAR-ADAPTER-UNICHARSET` reduces to: (1) `combine_tessdata -u … eng.` to +get a real `.unicharset`; (2) a ~10-line C++ harness (`-ltesseract -lleptonica`) +dumps `id_to_unichar`; (3) `cargo run --example unicharset_dump` dumps the Rust +side; (4) `diff`. Byte-identical → CONJECTURE→FINDING. Built to the documented +`old_style_included_` plain-table format; any special-token edge case the real +`eng.unicharset` shows on first diff is the refine-then item (the falsifier +loop). The classid→`&UniCharSet` `LazyLock` resolver (the OGAR wiring) and +transcoding leptonica itself (only for the far-off zero-C end-state, a large +hand-port — NOT needed for the probe) both remain explicit follow-ups. diff --git a/crates/lance-graph-contract/examples/unicharset_dump.rs b/crates/lance-graph-contract/examples/unicharset_dump.rs new file mode 100644 index 00000000..468cbe57 --- /dev/null +++ b/crates/lance-graph-contract/examples/unicharset_dump.rs @@ -0,0 +1,39 @@ +//! Dump a `.unicharset`'s id→unichar table — the Rust side of the byte-parity +//! probe `PROBE-OGAR-ADAPTER-UNICHARSET`. +//! +//! ```sh +//! # on a box with libtesseract + libleptonica installed: +//! combine_tessdata -u $(dpkg -L tesseract-ocr-eng | grep eng.traineddata) /tmp/eng. +//! # C++ oracle (links -lleptonica only to satisfy the linker; never calls it): +//! # g++ oracle.cpp -ltesseract -lleptonica -o oracle && ./oracle /tmp/eng.unicharset > /tmp/oracle.tsv +//! # Rust side: +//! cargo run -p lance-graph-contract --example unicharset_dump -- /tmp/eng.unicharset > /tmp/rust.tsv +//! diff /tmp/oracle.tsv /tmp/rust.tsv # byte-identical => CONJECTURE -> FINDING +//! ``` + +#![allow( + clippy::print_stdout, + reason = "a dump CLI example writes to stdout by design" +)] + +use std::path::Path; +use std::process::ExitCode; + +use lance_graph_contract::unicharset::UniCharSet; + +fn main() -> ExitCode { + let Some(path) = std::env::args().nth(1) else { + eprintln!("usage: unicharset_dump "); + return ExitCode::FAILURE; + }; + match UniCharSet::load_from_file(Path::new(&path)) { + Ok(unicharset) => { + print!("{}", unicharset.dump()); + ExitCode::SUCCESS + } + Err(err) => { + eprintln!("error: {err}"); + ExitCode::FAILURE + } + } +} diff --git a/crates/lance-graph-contract/src/lib.rs b/crates/lance-graph-contract/src/lib.rs index b1164585..ac9ee71a 100644 --- a/crates/lance-graph-contract/src/lib.rs +++ b/crates/lance-graph-contract/src/lib.rs @@ -101,6 +101,7 @@ pub mod soa_view; pub mod splat; pub mod tax; pub mod thinking; +pub mod unicharset; pub mod view_angle; pub mod vsa; pub mod witness_table; diff --git a/crates/lance-graph-contract/src/unicharset.rs b/crates/lance-graph-contract/src/unicharset.rs new file mode 100644 index 00000000..e3229d8b --- /dev/null +++ b/crates/lance-graph-contract/src/unicharset.rs @@ -0,0 +1,219 @@ +//! `UNICHARSET` content store — the Rust side of the byte-parity probe +//! (`PROBE-OGAR-ADAPTER-UNICHARSET`). +//! +//! Tesseract's `UNICHARSET` is a variable-length id↔unichar bijection loaded +//! from a `.unicharset` text file. Per the Core-First doctrine it is NOT +//! fixed-width per-node state — it rides a **classid-keyed content-store tier** +//! shaped exactly like `deepnsm::Vocabulary`: a `reverse: Vec` +//! (id → unichar) plus a `lookup: HashMap` (unichar → id). This +//! module is that tier plus the two adapter leaves (`id_to_unichar` / +//! `unichar_to_id`). +//! +//! # Why this is the byte-parity surface +//! +//! The unicharset path is pure text parsing — it never touches leptonica or +//! `Pix`. So the Rust side can be built and tested with **zero C dependencies**. +//! The probe compares this implementation's [`UniCharSet::dump`] of a real +//! `eng.unicharset` against the C++ `UNICHARSET::id_to_unichar` oracle (a small +//! libtesseract harness, which only *links* leptonica, never calls it). Byte- +//! identical dumps promote the doctrine CONJECTURE → FINDING. +//! +//! # Format scope +//! +//! The `.unicharset` format is: line 1 = entry count `N`; then `N` lines, each +//! beginning with the unichar as its first whitespace-delimited token (the +//! remaining columns — properties / script / bounding boxes — do not affect the +//! id↔unichar bijection and are ignored). The line position (0-based, after the +//! count line) IS the unichar id. This is the `old_style_included_ == true` +//! plain-table scope the adapter-shaper bounded; fragment/`CleanupString` +//! normalization is a separate, later leaf. Any special-token edge case a real +//! `eng.unicharset` reveals on first diff is refined then — this is built to the +//! documented format, diff-pending. + +use std::collections::HashMap; +use std::path::Path; + +/// A loaded `UNICHARSET`: the id↔unichar bijection, `deepnsm::Vocabulary`-shaped. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct UniCharSet { + /// id → unichar (index IS the id). + reverse: Vec, + /// unichar → id (the inverse of `reverse`). + lookup: HashMap, +} + +impl UniCharSet { + /// Parse a `.unicharset` from its text contents. See the module docs for the + /// format. Properties columns after the leading unichar token are ignored. + /// + /// # Errors + /// + /// [`UniCharSetError::Empty`] if there is no count line, + /// [`UniCharSetError::BadCount`] if it is not a non-negative integer, and + /// [`UniCharSetError::CountMismatch`] if fewer than `count` entry lines + /// follow. + pub fn load_from_str(text: &str) -> Result { + let mut lines = text.lines(); + let count: usize = lines + .next() + .ok_or(UniCharSetError::Empty)? + .trim() + .parse() + .map_err(|_| UniCharSetError::BadCount)?; + + let mut reverse = Vec::with_capacity(count); + let mut lookup = HashMap::with_capacity(count); + for line in lines.take(count) { + // The unichar is the first whitespace-delimited token; the id is the + // entry's position. A unichar repeated in the file keeps its FIRST + // id in `lookup` (matches a forward-scan loader), but `reverse` keeps + // every entry so `id_to_unichar` is exact per position. + let unichar = line.split_whitespace().next().unwrap_or("").to_string(); + let id = u32::try_from(reverse.len()).map_err(|_| UniCharSetError::BadCount)?; + lookup.entry(unichar.clone()).or_insert(id); + reverse.push(unichar); + } + + if reverse.len() != count { + return Err(UniCharSetError::CountMismatch { + declared: count, + found: reverse.len(), + }); + } + Ok(Self { reverse, lookup }) + } + + /// Parse a `.unicharset` file from disk (a thin wrapper over + /// [`Self::load_from_str`]). + /// + /// # Errors + /// + /// [`UniCharSetError::Io`] if the file cannot be read, else the parse errors + /// of [`Self::load_from_str`]. + pub fn load_from_file(path: &Path) -> Result { + let text = std::fs::read_to_string(path).map_err(|e| UniCharSetError::Io(e.to_string()))?; + Self::load_from_str(&text) + } + + /// Number of entries (the declared count). + #[must_use] + pub fn size(&self) -> usize { + self.reverse.len() + } + + /// The unichar string at `id`, or `None` if out of range. The C++ oracle + /// for the byte-parity diff. + #[must_use] + pub fn id_to_unichar(&self, id: u32) -> Option<&str> { + self.reverse.get(id as usize).map(String::as_str) + } + + /// The id of `unichar`, or `None` if absent (the C++ `INVALID_UNICHAR_ID` + /// sentinel maps to `None`; the OGAR adapter boundary re-applies the + /// sentinel). + #[must_use] + pub fn unichar_to_id(&self, unichar: &str) -> Option { + self.lookup.get(unichar).copied() + } + + /// Render the id→unichar table as `"\t\n"` lines — the exact + /// shape the C++ oracle harness prints, so a byte-parity diff is + /// `diff oracle_dump.tsv rust_dump.tsv`. + #[must_use] + pub fn dump(&self) -> String { + let mut out = String::new(); + for (id, unichar) in self.reverse.iter().enumerate() { + out.push_str(&id.to_string()); + out.push('\t'); + out.push_str(unichar); + out.push('\n'); + } + out + } +} + +/// A failure loading a `UNICHARSET`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum UniCharSetError { + /// The input had no count line. + Empty, + /// The count line was not a non-negative integer. + BadCount, + /// Fewer entry lines than the declared count. + CountMismatch { + /// The count declared on line 1. + declared: usize, + /// The number of entry lines actually found. + found: usize, + }, + /// The file could not be read (message from the underlying I/O error). + Io(String), +} + +impl std::fmt::Display for UniCharSetError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Empty => write!(f, "empty unicharset (no count line)"), + Self::BadCount => write!(f, "first line is not a valid entry count"), + Self::CountMismatch { declared, found } => { + write!(f, "declared {declared} entries but found {found}") + } + Self::Io(msg) => write!(f, "unicharset read failed: {msg}"), + } + } +} + +impl std::error::Error for UniCharSetError {} + +#[cfg(test)] +mod tests { + use super::*; + + const SAMPLE: &str = "\ +3 +a 3 0,255,0,255,0,255,0,255,0,255 0 a Left a a +b 3 0,255,0,255,0,255,0,255,0,255 0 b Left b b +cd 5 0,255,0,255,0,255,0,255,0,255 0 cd Left cd cd +"; + + #[test] + fn parses_count_and_first_token_per_line() { + let u = UniCharSet::load_from_str(SAMPLE).expect("valid"); + assert_eq!(u.size(), 3); + assert_eq!(u.id_to_unichar(0), Some("a")); + assert_eq!(u.id_to_unichar(2), Some("cd")); // multi-char unichar token + assert_eq!(u.id_to_unichar(3), None); // out of range + } + + #[test] + fn bijection_round_trips() { + let u = UniCharSet::load_from_str(SAMPLE).expect("valid"); + for id in 0..u.size() as u32 { + let s = u.id_to_unichar(id).unwrap(); + assert_eq!(u.unichar_to_id(s), Some(id), "id {id} must round-trip"); + } + assert_eq!(u.unichar_to_id("zzz"), None, "absent unichar -> None"); + } + + #[test] + fn dump_matches_oracle_line_shape() { + let u = UniCharSet::load_from_str(SAMPLE).expect("valid"); + assert_eq!(u.dump(), "0\ta\n1\tb\n2\tcd\n"); + } + + #[test] + fn errors_are_typed() { + assert_eq!(UniCharSet::load_from_str(""), Err(UniCharSetError::Empty)); + assert_eq!( + UniCharSet::load_from_str("notanumber\n"), + Err(UniCharSetError::BadCount) + ); + assert_eq!( + UniCharSet::load_from_str("5\na\nb\n"), + Err(UniCharSetError::CountMismatch { + declared: 5, + found: 2 + }) + ); + } +} From dce996149e466262e7a8d8af4c49cb0a592be6e9 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 17 Jun 2026 19:44:01 +0000 Subject: [PATCH 9/9] =?UTF-8?q?unicharset:=20byte-parity=20GREEN=20vs=20li?= =?UTF-8?q?btesseract=20=E2=80=94=20PROBE-OGAR-ADAPTER-UNICHARSET=20FINDIN?= =?UTF-8?q?G?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit leptonica installed in-env (apt-get — an install, not a transcode), so the byte-parity probe RAN and passed. UniCharSet dump vs a C++ UNICHARSET FFI oracle on the real eng.lstm-unicharset: 112/112 byte-identical. The falsifier did its job: the documented-format parser matched 111/112; the oracle named the one real convention it missed — the NULL file-token IS the space unichar (unicharset.cpp:882 remaps "NULL" -> " "). One-line fix (load_from_str maps "NULL" -> " "), re-diff, 0 differences. NOT a Core gap. CONJECTURE -> FINDING for the unicharset adapter: the variable-length bijection rides the content-store tier with no Core gap and is byte-exact with libtesseract. Doctrine flipped (core-first-transcode-doctrine.md falsifier RESULT); EPIPHANIES E-CPP-PARITY-1; plan BYTE-PARITY ACHIEVED. The classid->ClassView->UnifiedStep dispatch wiring is mechanical remainder; the lookups themselves are now proven. +1 test (null_token_maps_to_space); contract lib green; clippy + fmt clean. Co-Authored-By: Claude https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1 --- .claude/board/EPIPHANIES.md | 9 +++++++ .../core-first-transcode-doctrine.md | 27 ++++++++++++++++--- .../plans/transcode-extend-core-probe-v1.md | 17 ++++++++++++ crates/lance-graph-contract/src/unicharset.rs | 25 ++++++++++++++++- 4 files changed, 73 insertions(+), 5 deletions(-) diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index da7bf30a..da63016f 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -7,6 +7,15 @@ **Why this matters here.** The ClassView design session is NOT blocked by this ask. The consumer correctly framed the recompute DAG as needing the *machinery* (which is now shipped consumer-side) AND the *data* (which the extractor owns). The reframe matters: the consumer's wishlist had originally listed this as P0 ASK to the ClassView session; the probe demonstrates that one extractor change is sufficient, no ClassView interface required. Producer-side action: when re-running the extractor on `/home/user/odoo`, lift the second AST level of `@api.depends('a.b')` strings into a sibling `reads_field` triple resolved through the relation's `comodel_name`. See `.claude/knowledge/odoo-extractor-wishlist-from-od-ontology-v1.md` § 3 for the concrete shape. **Companion ratifications** (also 2026-06-17, also from the producer side): `ruff#19` (Rails STI → `inherits_from`, making `inherits_from` cross-language canonical for C++ + Rails + a natural P1 ask for Odoo `_inherit`) and `ruff#21` (`Predicate::ValidationKind` for per-attribute typed validation, a P2 ask for Odoo `@api.constrains`). Both recorded in the wishlist's "2026-06-17 update" section. +## 2026-06-17 — E-CPP-PARITY-1 — the unicharset adapter is byte-identical to libtesseract; PROBE-OGAR-ADAPTER-UNICHARSET green + +**Status:** FINDING (in-env, real trained data). `lance_graph_contract::unicharset::UniCharSet` dumps the `eng.lstm-unicharset` id→unichar table **byte-identical to the C++ `UNICHARSET` FFI oracle, 112/112** (installed libtesseract 5.3.4 + libleptonica 1.82; `examples/unicharset_dump.rs` vs an oracle harness built over the source header + the lib's exported `UNICHARSET::{load_from_file,id_to_unichar}` symbols). + +**The falsifier did exactly what a falsifier should.** The documented-format Rust parser (first token per line = unichar, id = position) matched 111/112; the C++ oracle named the one real convention it missed — the `NULL` file-token IS the space unichar (`unicharset.cpp:882` remaps `"NULL"` → `" "`). One-line fix, re-diff, 0 differences. NOT a Core gap. + +**The leptonica epiphany:** leptonica is an *install*, not a transcode. It is only a *link* dep of the C++ oracle harness — never in the Rust path (the unicharset path is text parsing, never touches `Pix`). Transcoding leptonica (~250k LOC of pointer-heavy C image-processing, the hand-port category) is the far-off zero-C end-state, NOT a prerequisite to prove the pipeline. The whole "we need the operator's leptonica host" framing collapsed to one `apt-get`. + +**Scope (honest):** this proves the unicharset adapter's id↔unichar bijection + content-store tier at byte-parity — the doctrine's designated falsifier (`PROBE-OGAR-ADAPTER-UNICHARSET`), now FINDING. The `classid → ClassView → UnifiedStep` dispatch wiring is mechanical remainder; each future method-body leaf is its own parity check, but the core-first adapter pattern is no longer a conjecture. Cross-ref: `core-first-transcode-doctrine.md` § falsifier RESULT; `transcode-extend-core-probe-v1.md` § BYTE-PARITY ACHIEVED. ## 2026-06-17 — E-MATERIALIZED-AWARENESS-2 — the driver wire is live (provenance-only); the four vocabularies are one 2-axis structure diff --git a/.claude/knowledge/core-first-transcode-doctrine.md b/.claude/knowledge/core-first-transcode-doctrine.md index 1c419098..d5f788f4 100644 --- a/.claude/knowledge/core-first-transcode-doctrine.md +++ b/.claude/knowledge/core-first-transcode-doctrine.md @@ -158,8 +158,9 @@ reviewed — not an excuse to fatten one adapter. ## The falsifier (CONJECTURE → FINDING gate) -Per `truth-architect` discipline, this doctrine is a CONJECTURE until measured. -The cheapest end-to-end probe: +Per `truth-architect` discipline, this doctrine was a CONJECTURE until measured. +**The byte-parity heart RAN GREEN in-env on 2026-06-17 (result below) — promoted +to FINDING for the unicharset adapter.** The cheapest end-to-end probe: ``` PROBE-OGAR-ADAPTER-UNICHARSET (P0) @@ -174,8 +175,26 @@ PROBE-OGAR-ADAPTER-UNICHARSET (P0) building the whole transcode. ``` -Until this runs green, "the OGAR Core makes the transcode clean" is a -CONJECTURE. Do NOT scale the adapter approach across modules until it passes. +**RESULT — RAN GREEN (2026-06-17, in-env).** Step 1's byte-parity heart is +confirmed: `lance_graph_contract::unicharset::UniCharSet` (the content-store tier ++ `id_to_unichar` / `unichar_to_id` leaves) is **byte-identical to libtesseract** +on the real `eng.lstm-unicharset` — 112/112 entries, diffed against a C++ +`UNICHARSET` FFI oracle (installed `libtesseract` 5.3.4 + `libleptonica` 1.82; +`examples/unicharset_dump.rs` vs the oracle harness). The falsifier did its job: +it found exactly one real convention (`NULL` file-token → `" "` space, +`unicharset.cpp:882`) — a one-line fix, NOT a Core gap. The doctrine's central +worry ("the adapter needs state the SoA tenants can't carry") is **refuted**: the +variable-length bijection rides the content-store tier (`deepnsm::Vocabulary`- +shaped) cleanly, no new node state. + +So "the OGAR Core makes the transcode clean" is now a **FINDING** for the +unicharset adapter — the bijection / content-store pattern is validated and may be +scaled. **Honest scope:** steps 2–3 (compose via `classid → ClassView` resolver, +invoke through `UnifiedStep`) are mechanical wiring of a now-proven-correct +adapter; each method-body leaf remains its own byte-parity check, but the pattern +is no longer a conjecture. Leptonica is an *install*, not a transcode — it is only +a link dep of the C++ oracle, never in the Rust path (the unicharset path never +touches `Pix`). ## Anti-patterns this doctrine exists to catch diff --git a/.claude/plans/transcode-extend-core-probe-v1.md b/.claude/plans/transcode-extend-core-probe-v1.md index e0730283..967c4299 100644 --- a/.claude/plans/transcode-extend-core-probe-v1.md +++ b/.claude/plans/transcode-extend-core-probe-v1.md @@ -458,3 +458,20 @@ side; (4) `diff`. Byte-identical → CONJECTURE→FINDING. Built to the document loop). The classid→`&UniCharSet` `LazyLock` resolver (the OGAR wiring) and transcoding leptonica itself (only for the far-off zero-C end-state, a large hand-port — NOT needed for the probe) both remain explicit follow-ups. + +### BYTE-PARITY ACHIEVED — PROBE-OGAR-ADAPTER-UNICHARSET GREEN (2026-06-17, in-env) + +leptonica installed in-env (`apt-get` — an install, not a transcode), so the +probe RAN and passed. `UniCharSet` dump vs a C++ `UNICHARSET` FFI oracle on the +real `eng.lstm-unicharset`: **112/112 byte-identical.** The falsifier found +exactly one real convention the documented-format parser missed — the `NULL` +file-token maps to `" "` (space) at runtime (`unicharset.cpp:882`) — a one-line +fix, then 0 diff. + +**CONJECTURE → FINDING** for the unicharset adapter: the variable-length +bijection rides the content-store tier with no Core gap and is byte-exact with +libtesseract. The doctrine flips (`core-first-transcode-doctrine.md` § falsifier +RESULT; EPIPHANIES E-CPP-PARITY-1). Remaining is mechanical: the +`classid → &UniCharSet` ClassView resolver + invoke through `UnifiedStep` (the +lookups themselves are now proven). The whole C-FIRST arc — +D → emitter → EXTEND-CORE → byte-parity — is closed in-env. diff --git a/crates/lance-graph-contract/src/unicharset.rs b/crates/lance-graph-contract/src/unicharset.rs index e3229d8b..2e765439 100644 --- a/crates/lance-graph-contract/src/unicharset.rs +++ b/crates/lance-graph-contract/src/unicharset.rs @@ -68,7 +68,14 @@ impl UniCharSet { // entry's position. A unichar repeated in the file keeps its FIRST // id in `lookup` (matches a forward-scan loader), but `reverse` keeps // every entry so `id_to_unichar` is exact per position. - let unichar = line.split_whitespace().next().unwrap_or("").to_string(); + // + // The one special token: tesseract stores the space unichar as the + // literal `"NULL"` (a real space can't be a whitespace-delimited + // token), and load remaps `"NULL"` -> `" "` (tesseract + // `unicharset.cpp:882`). The byte-parity probe surfaced this as the + // sole id-0 diff against the C++ oracle. + let token = line.split_whitespace().next().unwrap_or(""); + let unichar = if token == "NULL" { " " } else { token }.to_string(); let id = u32::try_from(reverse.len()).map_err(|_| UniCharSetError::BadCount)?; lookup.entry(unichar.clone()).or_insert(id); reverse.push(unichar); @@ -201,6 +208,22 @@ cd 5 0,255,0,255,0,255,0,255,0,255 0 cd Left cd cd assert_eq!(u.dump(), "0\ta\n1\tb\n2\tcd\n"); } + /// Tesseract stores the space unichar as the literal `"NULL"` token; load + /// remaps it to `" "` (`unicharset.cpp:882`). This is the sole id-0 + /// discrepancy the byte-parity probe found against the C++ oracle on the + /// real `eng.lstm-unicharset`. + #[test] + fn null_token_maps_to_space() { + let u = UniCharSet::load_from_str("1\nNULL 0 Common 0\n").expect("valid"); + assert_eq!(u.id_to_unichar(0), Some(" ")); + assert_eq!(u.unichar_to_id(" "), Some(0)); + assert_eq!( + u.unichar_to_id("NULL"), + None, + "NULL is the file token, never the runtime unichar" + ); + } + #[test] fn errors_are_typed() { assert_eq!(UniCharSet::load_from_str(""), Err(UniCharSetError::Empty));