Add V3 board epiphanies, data-shape etymology doc, and deepnsm gridlake examples

[AdaWorldAPI]

What

Adds V3-substrate board/knowledge content and three deepnsm example programs.

Board & knowledge

• .claude/board/EPIPHANIES.md — new V3 epiphany entries: ValueSchema substrate, semantic-kernel/RAG, rig chassis, tokenizer mint membrane, GraphRAG-as-vehicle, retrieval-vs-cognition, typed reasoning fanout, ARM induction organ, rs-graph-llm repatriation, Jina fulcrum, codec-fidelity-is-representation, dual-schema 0x1000, and think-atoms perturbation cascade.
• .claude/knowledge/data-shape-etymology.md — knowledge doc (8 epiphanies + a litmus battery).
• .claude/handovers/2026-07-02-visions-to-future-sessions.md — handover letter.

Examples (crates/deepnsm/examples/)

• gridlake_coca_wire.rs — gridlake-4096 ⟷ COCA-4096 wire spike.
• gridlake_spo_covariance.rs — cross-perturbation / covariance probe.
• gridlake_spo_ngrams.rs — real-SPO n-gram landing.

Verification

• cargo build --manifest-path crates/deepnsm/Cargo.toml --examples — all three examples compile clean.
• Board/knowledge changes are additive (append-only entries + new files); no existing entries modified.

🤖 Generated with Claude Code

[coderabbitai]

Warning

Review limit reached

@AdaWorldAPI, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4fbc8a17-df82-4bb9-8009-17653a12155e

📥 Commits

Reviewing files that changed from the base of the PR and between b8f6bc6 and 33b2f3b.

📒 Files selected for processing (3)

• crates/deepnsm/examples/gridlake_coca_wire.rs
• crates/deepnsm/examples/gridlake_spo_covariance.rs
• crates/deepnsm/examples/gridlake_spo_ngrams.rs

📝 Walkthrough

Walkthrough

This PR adds several new Markdown documentation files to .claude/board and .claude/knowledge recording architectural findings, rulings, and etymology doctrine, plus three new standalone Rust example binaries in crates/deepnsm/examples demonstrating COCA-rank-based grid landing, throughput sweeps, and spectral/covariance layout analysis. No exported/public entities change.

Changes

Workspace doctrine documents

Layer / File(s)	Summary
EPIPHANIES.md entries .claude/board/EPIPHANIES.md	Adds dated entries on THINK/DO unification, schema discriminator rulings, codec fidelity findings, rig-arm/AriGraph specs, adapter repatriation, cognition fanout, chassis/tokenizer rules, and GraphRAG reframes.
Visions-to-future-sessions handover .claude/handovers/2026-07-02-visions-to-future-sessions.md	New handover letter presenting five thematic visions, a torches checklist, and closing guidance for future sessions.
Data-shape etymology knowledge doc .claude/knowledge/data-shape-etymology.md	New doctrine document with eight sections on naming/provenance, masks, derivability, witness costs, ValueSchema resolution, homonyms, and a hat-trick test.

Gridlake Rust examples

Layer / File(s)	Summary
COCA token landing and throughput sweep crates/deepnsm/examples/gridlake_coca_wire.rs	New example landing tokenized COCA ranks into a 4096-cell grid with helix48/campq48 encoding and a large throughput sweep with checksum verification.
N-gram ingestion into grid crates/deepnsm/examples/gridlake_spo_ngrams.rs	New example ingesting tab-separated n-gram files into per-cell counts/truth weights, reporting spread metrics and top content cells.
Spectral and rank layout covariance probes crates/deepnsm/examples/gridlake_spo_covariance.rs	New example building a co-occurrence adjacency matrix, extracting eigenvectors via power iteration, and comparing rank vs spectral-layout edge-displacement statistics.

Estimated code review effort: 2 (Simple) | ~15 minutes

Possibly related PRs

• AdaWorldAPI/lance-graph#422: Both PRs modify .claude/board/EPIPHANIES.md with related epiphany entries.
• AdaWorldAPI/lance-graph#465: Overlaps on the same substrate palette/Hamming and kv-lance/rig architecture framing in EPIPHANIES.md.
• AdaWorldAPI/lance-graph#617: Extends related .claude/knowledge doctrine on ClassView/AST addressing referenced in this PR's docs.

Poem

A rabbit scribbles doctrine deep,
Epiphanies stacked before sleep,
Grid cells land where COCA ranks flow,
Eigenvectors spin, spectral lights glow.
Hop, hop — the knowledge base grows! 🐇📜

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main changes: V3 board epiphanies, the data-shape etymology document, and deepnsm gridlake examples.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b8f6bc6. Configure here.}

[cursor]

Empty landed slice index panic

Low Severity

The throughput benchmark always runs after token landing, but when no in-vocabulary tokens were landed landed stays empty. Using landed.len().max(1) only fixes modulo-by-zero; indexing landed[0] on an empty vector still panics, so the example can crash after a successful vocab load if every token is OOV.

 

^{Reviewed by Cursor Bugbot for commit b8f6bc6. Configure here.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b8f6bc69f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Measure the full codec in the throughput loop

In the release throughput scenario, the only post-loop observation is count, so the timed sum_truth and campq48 stores are dead from the program's perspective, and the loop also never recomputes helix48 even though the output labels the run as 48h+48pq encode each. This can make the reported codec throughput collapse to counter increments rather than the full helix+CAM_PQ work; include the codec bytes in the checksum/black_box and update helix48 in the measured path.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Fail fast when the co-occurrence graph is empty

When the default /tmp/sources/coca files are absent in a fresh checkout, this if let Ok silently skips both inputs and leaves edges empty; the example then runs the expensive eigensolver over an all-zero 4096² matrix and later divides by zero in edge_cov, producing NaN measurements instead of a usable failure. Please report missing inputs or abort when no edges were loaded before continuing to the spectral pass.

Useful? React with 👍 / 👎.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (7)

crates/deepnsm/examples/gridlake_spo_ngrams.rs (1)

29-35: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

No unit tests for rank_of/ingest.

Both are pure/parseable-input functions well-suited to focused #[cfg(test)] cases (e.g., a small synthetic TSV fixture) per repo guideline.

As per coding guidelines: "Add Rust unit tests alongside implementations via #[cfg(test)] modules; prefer focused scenarios over broad integration tests."

Also applies to: 37-70

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_spo_ngrams.rs` around lines 29 - 35, Add
focused Rust unit tests for the pure/parseable-input logic in rank_of and ingest
by introducing a #[cfg(test)] module alongside gridlake_spo_ngrams.rs. Cover
rank_of with a small synthetic Vocabulary/tokenization case and cover ingest
with a tiny TSV fixture so the behavior is validated without broad integration
setup. Use the existing rank_of and ingest functions as the entry points for the
tests.

Source: Coding guidelines

crates/deepnsm/examples/gridlake_spo_covariance.rs (3)

35-111: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

No unit tests for the linear-algebra helpers.

matvec/dot/normalize/eig/edge_cov are pure and independently testable (e.g., a small known adjacency matrix with a hand-computed eigenvector/covariance), per repo guideline.

As per coding guidelines: "Add Rust unit tests alongside implementations via #[cfg(test)] modules; prefer focused scenarios over broad integration tests."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_spo_covariance.rs` around lines 35 - 111,
Add focused Rust unit tests for the pure linear-algebra helpers in the same
module using a #[cfg(test)] mod. Cover matvec, dot, normalize, eig, and edge_cov
with small deterministic inputs (for example a tiny adjacency matrix and a
hand-checkable edge set) so the expected vector, normalization, eigenpair
behavior, and covariance outputs are verified directly.

Source: Coding guidelines

---

183-187: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

partial_cmp(...).unwrap() panics on NaN.

If an eigenvector component is ever NaN (e.g., degenerate deflation), sorting panics. Low likelihood given current normalization, but a partial_cmp(...).unwrap_or(Ordering::Equal) would make this robust against future changes.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_spo_covariance.rs` around lines 183 - 187,
The sorting in the eigenvector index preparation currently uses
partial_cmp(...).unwrap(), which can panic if evs[1] or evs[2] contains NaN.
Update the ex.sort_by and ey.sort_by comparisons in gridlake_spo_covariance to
handle non-comparable values safely, for example by falling back to a stable
default ordering instead of unwrapping. Keep the fix localized to the
eigenvector sorting logic around sem_pos initialization.

---

35-44: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Likely clippy::needless_range_loop hits.

The 0..N index loops in matvec, the eig deflation, and degree normalization index into slices by position; clippy typically flags this in favor of iterator-based access, which also helps eliminate bounds checks.

As per coding guidelines: "Run cargo clippy --all-targets --all-features to catch lint regressions in Rust code."

Also applies to: 63-83, 148-161

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_spo_covariance.rs` around lines 35 - 44, The
indexed `0..N` loops in `matvec` and the other slice-walking code in `eig`
deflation and degree normalization are likely triggering
`clippy::needless_range_loop`; refactor these paths to use iterator-based
traversal (`iter`, `iter_mut`, `zip`, `enumerate`) instead of manual indexing
into slices. Update the implementations in `matvec` and the corresponding logic
in `eig` and normalization helpers so they preserve behavior while avoiding
range-based indexing, then run `cargo clippy --all-targets --all-features` to
confirm the lint is cleared.

Source: Coding guidelines

crates/deepnsm/examples/gridlake_coca_wire.rs (3)

14-15: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

GRID = 4096 silently duplicates the library's VOCAB_SIZE.

Vocabulary::load caps ranks at its internal VOCAB_SIZE and Convert 1-based rank to 0-based index, cap at VOCAB_SIZE. This example hardcodes GRID = 4096 (and the literal string "VOCAB_SIZE=4096" at line 46) instead of deriving it from the crate. If the library's vocab size ever changes, grid[rank] indexing (line 75) and the throughput sweep would panic with an out-of-bounds access rather than failing loudly at a single, obvious point.

Consider exposing a public constant/accessor on Vocabulary and using it here instead of a duplicated magic number, to keep this contract explicit across the three gridlake examples that all repeat it.

Also applies to: 42-46, 57-77

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_coca_wire.rs` around lines 14 - 15, Replace
the duplicated magic vocab size in the gridlake example with the crate’s
authoritative value so the contract stays in sync. Update the constants and any
related checks in this example to derive the grid size from a public
`Vocabulary` constant/accessor instead of hardcoding `GRID = 4096` or
`"VOCAB_SIZE=4096"`, and make sure the `grid[rank]` indexing and throughput
sweep use that shared source of truth. Apply the same change consistently across
the other gridlake examples that repeat this value.

---

109-126: 🩺 Stability & Availability | 🔵 Trivial | 💤 Low value

.max(1) guard doesn't actually protect against an empty landed vec.

If landed is empty, landed.len().max(1) avoids a modulo-by-zero, but landed[0] on a zero-length vec still panics. Currently unreachable since the hardcoded grok text guarantees known tokens, but the guard reads as intentional protection it doesn't provide.

🛡️ Proposed fix

-    let landed: Vec<u16> = cells.iter().map(|&c| c as u16).collect();
+    let landed: Vec<u16> = cells.iter().map(|&c| c as u16).collect();
+    if landed.is_empty() {
+        eprintln!("no known tokens landed; skipping throughput sweep");
+        return;
+    }
     let rows: u64 = 300_000_000;
     let t = Instant::now();
     let mut i = 0usize;
     for _ in 0..rows {
-        let rank = landed[i % landed.len().max(1)] as usize;
+        let rank = landed[i % landed.len()] as usize;

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_coca_wire.rs` around lines 109 - 126, The
throughput sweep in gridlake_coca_wire uses landed.len().max(1) in the index
expression, but that does not prevent a panic when landed is empty because
landed[0] can still be reached. Update the loop around landed, rows, and the
rank selection to explicitly handle the empty Vec<u16> case before indexing, or
skip the sweep entirely when landed.is_empty(), so the logic in the sweep is
truly safe instead of relying on a misleading guard.

---

25-40: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

No #[cfg(test)] coverage for land.

land is pure and deterministic (given a palette), a good candidate for a focused unit test verifying helix48/campq48 output for a known input, per repo guideline.

As per coding guidelines: "Add Rust unit tests alongside implementations via #[cfg(test)] modules; prefer focused scenarios over broad integration tests."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_coca_wire.rs` around lines 25 - 40, Add
focused #[cfg(test)] unit coverage for land in gridlake_coca_wire.rs. Since land
is pure and deterministic given the palette, create a small test module next to
land that exercises a known word/palette input and asserts the expected campq48
and helix48 results, using land and Cell as the key symbols to locate the
behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/deepnsm/examples/gridlake_spo_covariance.rs`:
- Around line 88-111: Handle the empty-ngram case in gridlake_spo_covariance by
making the ingest path report missing data like gridlake_spo_ngrams::ingest and
by guarding edge_cov against sw == 0.0. Update edge_cov to return a safe default
or skip metric computation when edges is empty, and ensure the caller around the
ingest/metric flow does not propagate NaN into the verdict; use the edge_cov and
ingest symbols to locate both the division point and the no-op ingestion path.

---

Nitpick comments:
In `@crates/deepnsm/examples/gridlake_coca_wire.rs`:
- Around line 14-15: Replace the duplicated magic vocab size in the gridlake
example with the crate’s authoritative value so the contract stays in sync.
Update the constants and any related checks in this example to derive the grid
size from a public `Vocabulary` constant/accessor instead of hardcoding `GRID =
4096` or `"VOCAB_SIZE=4096"`, and make sure the `grid[rank]` indexing and
throughput sweep use that shared source of truth. Apply the same change
consistently across the other gridlake examples that repeat this value.
- Around line 109-126: The throughput sweep in gridlake_coca_wire uses
landed.len().max(1) in the index expression, but that does not prevent a panic
when landed is empty because landed[0] can still be reached. Update the loop
around landed, rows, and the rank selection to explicitly handle the empty
Vec<u16> case before indexing, or skip the sweep entirely when
landed.is_empty(), so the logic in the sweep is truly safe instead of relying on
a misleading guard.
- Around line 25-40: Add focused #[cfg(test)] unit coverage for land in
gridlake_coca_wire.rs. Since land is pure and deterministic given the palette,
create a small test module next to land that exercises a known word/palette
input and asserts the expected campq48 and helix48 results, using land and Cell
as the key symbols to locate the behavior.

In `@crates/deepnsm/examples/gridlake_spo_covariance.rs`:
- Around line 35-111: Add focused Rust unit tests for the pure linear-algebra
helpers in the same module using a #[cfg(test)] mod. Cover matvec, dot,
normalize, eig, and edge_cov with small deterministic inputs (for example a tiny
adjacency matrix and a hand-checkable edge set) so the expected vector,
normalization, eigenpair behavior, and covariance outputs are verified directly.
- Around line 183-187: The sorting in the eigenvector index preparation
currently uses partial_cmp(...).unwrap(), which can panic if evs[1] or evs[2]
contains NaN. Update the ex.sort_by and ey.sort_by comparisons in
gridlake_spo_covariance to handle non-comparable values safely, for example by
falling back to a stable default ordering instead of unwrapping. Keep the fix
localized to the eigenvector sorting logic around sem_pos initialization.
- Around line 35-44: The indexed `0..N` loops in `matvec` and the other
slice-walking code in `eig` deflation and degree normalization are likely
triggering `clippy::needless_range_loop`; refactor these paths to use
iterator-based traversal (`iter`, `iter_mut`, `zip`, `enumerate`) instead of
manual indexing into slices. Update the implementations in `matvec` and the
corresponding logic in `eig` and normalization helpers so they preserve behavior
while avoiding range-based indexing, then run `cargo clippy --all-targets
--all-features` to confirm the lint is cleared.

In `@crates/deepnsm/examples/gridlake_spo_ngrams.rs`:
- Around line 29-35: Add focused Rust unit tests for the pure/parseable-input
logic in rank_of and ingest by introducing a #[cfg(test)] module alongside
gridlake_spo_ngrams.rs. Cover rank_of with a small synthetic
Vocabulary/tokenization case and cover ingest with a tiny TSV fixture so the
behavior is validated without broad integration setup. Use the existing rank_of
and ingest functions as the entry points for the tests.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 52fe1af2-8e92-4552-9215-ffb36300a878

📥 Commits

Reviewing files that changed from the base of the PR and between 7a5c066 and b8f6bc6.

📒 Files selected for processing (6)

• .claude/board/EPIPHANIES.md
• .claude/handovers/2026-07-02-visions-to-future-sessions.md
• .claude/knowledge/data-shape-etymology.md
• crates/deepnsm/examples/gridlake_coca_wire.rs
• crates/deepnsm/examples/gridlake_spo_covariance.rs
• crates/deepnsm/examples/gridlake_spo_ngrams.rs

[coderabbitai]

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Silent divide-by-zero when no ngram data is present.

If the (uncommitted, licensed) ngram files aren't found at the given path, the ingest closure quietly no-ops (no message, unlike the sibling gridlake_spo_ngrams.rs::ingest, which prints "(missing {} — skipped)"), leaving edges empty. edge_cov then divides mx/my by sw == 0.0 (lines 98-99), producing NaN for the rank/spectral projection metrics and the final verdict — with no indication anything went wrong. This is likely to be the default first-run experience for anyone without the corpus.

🛡️ Proposed fix

     let mut ingest = |file: &str, ca: usize, cb: usize, minf: usize| {
-        if let Ok(t) = std::fs::read_to_string(dir.join(file)) {
+        if let Ok(t) = std::fs::read_to_string(dir.join(file)) {
             for line in t.lines() {
                 ...
             }
+        } else {
+            eprintln!("  ({} not found — skipped)", dir.join(file).display());
         }
     };

 fn edge_cov(edges: &[(usize, usize, f32)], pos: &[(f32, f32)]) -> (f32, f32, f32, f32) {
+    if edges.is_empty() {
+        return (0.0, 0.0, 0.0, 0.0);
+    }
     let mut sw = 0f64;

Also applies to: 125-142

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/deepnsm/examples/gridlake_spo_covariance.rs` around lines 88 - 111,
Handle the empty-ngram case in gridlake_spo_covariance by making the ingest path
report missing data like gridlake_spo_ngrams::ingest and by guarding edge_cov
against sw == 0.0. Update edge_cov to return a safe default or skip metric
computation when edges is empty, and ensure the caller around the ingest/metric
flow does not propagate NaN into the verdict; use the edge_cov and ingest
symbols to locate both the division point and the no-op ingestion path.