onebrc/lane-j: wire GridBatch through ndarray MultiLaneColumn (D-DNV-1)

[AdaWorldAPI]

What

First code step of the DeepNSM→V3 convergence plan (#640): D-DNV-1, the gridlake carrier foundation. The proven 64×64 gridlake batch table (E-1BRC-GRIDLAKE-SWEETSPOT-1, #635) now renders its four accumulator columns through ndarray::simd::MultiLaneColumn — the SoA-contract carrier — instead of being an opaque set of parallel Vecs.

The insight this lands (from E-V3-DEEPNSM-IS-THE-ENCODER-NOT-A-MIGRATION-1): the gridlake batch table is not a bespoke struct — it is typed lanes over one carrier, the same MultiLaneColumn the COCA cognitive Cell (helix48/campq48/count/truth, crates/deepnsm/examples/gridlake_coca_wire.rs) composes from. Wire, don't invent.

Change

• crates/onebrc-probe/Cargo.toml — the lane-j feature now pulls ndarray (default-features = false, std), the same fork dep lane-b already uses.
• crates/onebrc-probe/src/lane_j.rs — GridBatch::as_gridlake_columns() -> Result<GridlakeColumns, ()>: a little-endian reading (zero re-layout) of the four accumulators onto the integer lanes ndarray added for exactly this — min/max ride I32x16 (iter_i32x16 "min/max tile columns"), sum rides I64x8 (iter_i64x8 "running sums"), count (non-negative accumulator) is widened u32 → u64 onto U64x8. Returns Err(()) on a mis-aligned grid, mirroring MultiLaneColumn::new's own 64-byte contract.

Verification

cargo test --manifest-path crates/onebrc-probe/Cargo.toml --features lane-j — 2/2 green:

• gridlake_batch_rides_multilane_column_losslessly — the LE roundtrip is cell-for-cell exact against the source mins/maxs/sums/counts (populated across lane-boundary cells 15|16 and the tile edge 4095); typed-lane views wired (len_i32x16/iter_i32x16 == grid/16, i64x8/u64x8 == grid/8).
• gridlake_carrier_rejects_unaligned_grid — a grid not a multiple of 16 is refused, not silently mis-carried.

lane_j.rs is fmt-clean and clippy-clean (the one result_unit_err carries #[allow] mirroring the MultiLaneColumn::new contract it passes through). Pre-existing fmt/clippy debt in sibling probe lanes (lane_s/lane_t) is left untouched — out of scope.

Scope

Read-side carrier only — no hot-path change, no new tenant, no ENVELOPE_LAYOUT_VERSION bump. STATUS_BOARD carries the deepnsm-v3-convergence-v1 section with D-DNV-1 = In PR and D-DNV-2..4 (SPO→CausalEdge64+2³ / arm-discovery leg / episodic-witness tenant) queued and gated per the plan.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MLBnPuScZy6w9di2QEjsXM

---

Generated by Claude Code

Summary by CodeRabbit

• New Features

  • Expanded the lane-j data path to support an additional vectorized array backend.
  • Added a new way to export batch results into a columnar format for faster downstream processing.
  • Included new validation so mismatched batch sizes are rejected instead of producing invalid output.

• Tests

  • Added coverage for round-trip conversion and alignment checks.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds SIMD gridlake carrier support to onebrc-probe: the lane-j feature now pulls in the ndarray dependency, GridBatch becomes public and gains an as_gridlake_columns() method producing a new GridlakeColumns struct of MultiLaneColumn fields, with roundtrip and alignment tests. A status board entry documents related deliverables.

Changes

Gridlake SIMD Carrier and Status Board

Layer / File(s)	Summary
Enable ndarray dependency crates/onebrc-probe/Cargo.toml	lane-j feature now also enables the dep:ndarray optional dependency.
GridBatch gridlake carrier rendering crates/onebrc-probe/src/lane_j.rs	GridBatch is made public, imports MultiLaneColumn, adds GridlakeColumns struct and as_gridlake_columns() method encoding accumulators as LE byte-backed columns, with roundtrip and unaligned-grid tests.
Status board entry .claude/board/STATUS_BOARD.md	Adds deepnsm-v3-convergence-v1 section with D-DNV-1..D-DNV-4 deliverables, statuses, and blockers.

Estimated code review effort: 2 (Simple) | ~12 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test
  participant GridBatch
  participant MultiLaneColumn

  Test->>GridBatch: as_gridlake_columns()
  GridBatch->>GridBatch: encode mins/maxs/sums as LE i32/i64 bytes
  GridBatch->>GridBatch: widen counts u32 to u64 LE bytes
  GridBatch->>MultiLaneColumn: new(bytes) for each field
  MultiLaneColumn-->>GridBatch: Ok(column) or alignment error
  GridBatch-->>Test: Result<GridlakeColumns, ()>

Loading

Possibly related PRs

• AdaWorldAPI/lance-graph#635: Extends the same Lane J grid/batch pipeline code with the new GridlakeColumns carrier and rendering method.

Poem

A rabbit hopped through columns four,
mins and maxs and sums galore 🐇
Bytes aligned in little-end grace,
sixteen-wide lanes fall into place.
Board updated, tests all green—
the tidiest burrow you've ever seen!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: wiring GridBatch through ndarray MultiLaneColumn in lane-j.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

crates/onebrc-probe/src/lane_j.rs (2)

218-238: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Three near-identical serialization helpers — consider a generic implementation.

col_i32, col_i64, and col_u64_from_u32 differ only in element width/conversion. A small generic helper (e.g., over IntoIterator + a byte-writer closure) would remove the duplication and reduce future drift risk if a fourth lane width is added.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/onebrc-probe/src/lane_j.rs` around lines 218 - 238, The three
serialization helpers col_i32, col_i64, and col_u64_from_u32 are near-identical
and should be consolidated. Refactor the MultiLaneColumn byte-building logic
into a shared generic helper used by these functions, with the type-specific
conversion handled via a small closure or trait-based adapter. Keep the existing
function names as thin wrappers if needed, but make the byte serialization and
Arc::from/ MultiLaneColumn::new flow live in one place to prevent drift when
adding more lane widths.

---

137-137: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Keep GridBatch crate-private unless you add an external constructor. GridlakeColumns is the only new public carrier here; GridBatch has no downstream entry point, so pub(crate) would keep the API surface smaller.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/onebrc-probe/src/lane_j.rs` at line 137, `GridBatch` is exposed as
public without any external construction path, so reduce its visibility to
crate-private in the `GridBatch` definition unless you also add a public
constructor or other downstream entry point. Update the `GridBatch` item in
`lane_j` to match the intended API surface, and keep `GridlakeColumns` as the
only new public carrier.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/board/STATUS_BOARD.md:
- Around line 1-11: The STATUS_BOARD.md update breaks the append-only ledger
convention by inserting a new section at the top and effectively reordering
prior status entries. Update the board by appending a dated correction or new
status entry at the end instead of changing the existing history in place, and
keep the existing entries and provenance in their original order; use the
STATUS_BOARD heading and the existing D-id rows as the anchors when making the
append-only fix.

In `@crates/onebrc-probe/src/lane_j.rs`:
- Line 217: The `as_gridlake_columns()` method currently returns
`Result<GridlakeColumns, ()>`, which bypasses the repo’s snafu error pattern and
forces a clippy allow. Replace the unit error with a proper `snafu`-derived
error type in the `lane_j` module, such as a `GridlakeCarrierError` variant that
captures the relevant alignment context (for example, the grid size or column
details) instead of propagating `MultiLaneColumn::new` as `Err(())`. Update the
`as_gridlake_columns` signature and its call sites to surface this richer error
consistently with the project’s Rust error conventions.

---

Nitpick comments:
In `@crates/onebrc-probe/src/lane_j.rs`:
- Around line 218-238: The three serialization helpers col_i32, col_i64, and
col_u64_from_u32 are near-identical and should be consolidated. Refactor the
MultiLaneColumn byte-building logic into a shared generic helper used by these
functions, with the type-specific conversion handled via a small closure or
trait-based adapter. Keep the existing function names as thin wrappers if
needed, but make the byte serialization and Arc::from/ MultiLaneColumn::new flow
live in one place to prevent drift when adding more lane widths.
- Line 137: `GridBatch` is exposed as public without any external construction
path, so reduce its visibility to crate-private in the `GridBatch` definition
unless you also add a public constructor or other downstream entry point. Update
the `GridBatch` item in `lane_j` to match the intended API surface, and keep
`GridlakeColumns` as the only new public carrier.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: a5b48c0e-3b7e-42a1-b2c7-ab57ee068968

📥 Commits

Reviewing files that changed from the base of the PR and between 70af76f and 8d98351.

📒 Files selected for processing (3)

• .claude/board/STATUS_BOARD.md
• crates/onebrc-probe/Cargo.toml
• crates/onebrc-probe/src/lane_j.rs

[coderabbitai]

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Preserve the board's append-only history.

This new section is inserted at the top, which conflicts with the append-only ledger convention and weakens provenance. Please append a dated entry/correction instead of reordering prior status entries.

Based on learnings: files under .claude/board/*.md are append-only ledgers; do not rewrite or edit prior entries in place.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/board/STATUS_BOARD.md around lines 1 - 11, The STATUS_BOARD.md
update breaks the append-only ledger convention by inserting a new section at
the top and effectively reordering prior status entries. Update the board by
appending a dated correction or new status entry at the end instead of changing
the existing history in place, and keep the existing entries and provenance in
their original order; use the STATUS_BOARD heading and the existing D-id rows as
the anchors when making the append-only fix.

Source: Learnings

[coderabbitai]

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Unit error type bypasses the repo's snafu convention.

as_gridlake_columns() returns Result<GridlakeColumns, ()> and needs a #[allow(clippy::result_unit_err)] to silence the resulting lint. The repo's Rust guideline calls for reusing snafu error patterns instead of a bare () error, which also loses diagnostic context (e.g., which column/grid size failed alignment) that a proper error variant could carry.

As per coding guidelines, "Format Rust code with cargo fmt --all; keep modules and functions snake_case, types PascalCase, and reuse snafu error patterns".

♻️ Suggested direction

-    #[allow(clippy::result_unit_err)] // pass-through of MultiLaneColumn::new's Result<_, ()> alignment contract
-    pub fn as_gridlake_columns(&self) -> Result<GridlakeColumns, ()> {
+    pub fn as_gridlake_columns(&self) -> Result<GridlakeColumns, GridlakeCarrierError> {

with a snafu-derived GridlakeCarrierError (e.g., UnalignedGridSnafu { grid: usize }) surfaced instead of swallowing MultiLaneColumn::new's Err(()).

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/onebrc-probe/src/lane_j.rs` at line 217, The `as_gridlake_columns()`
method currently returns `Result<GridlakeColumns, ()>`, which bypasses the
repo’s snafu error pattern and forces a clippy allow. Replace the unit error
with a proper `snafu`-derived error type in the `lane_j` module, such as a
`GridlakeCarrierError` variant that captures the relevant alignment context (for
example, the grid size or column details) instead of propagating
`MultiLaneColumn::new` as `Err(())`. Update the `as_gridlake_columns` signature
and its call sites to surface this richer error consistently with the project’s
Rust error conventions.

Source: Coding guidelines