feat(workspace): stand up the pinax workspace and implement lexis - #10
Merged
Conversation
added 2 commits
August 15, 2026 17:57
Four-crate workspace per Decision 14 (lexis -> hypomnema -> phylaxis -> pinax), root Cargo.toml carrying workspace.package/dependencies/lints, and the standard root files (deny.toml copied from heurema, clippy.toml, minimal rustfmt.toml, .config/nextest.toml). hypomnema, phylaxis, and pinax are conforming empty library crates that reserve their position in the locked dependency graph; only lexis is implemented. lexis implements Decision 5's strict six-type system: SqlType (the fixed INTEGER/REAL/TEXT/BLOB/BOOLEAN/DATETIME set, no NUMERIC/DECIMAL/DATE/ TIME/JSON), Value (the runtime value enum, NULL as a first-class variant for SQL three-valued logic), validated newtypes (RealValue rejects NaN at the insert-bind boundary, TextMaxLen and identifiers validate via TryFrom), schema DDL vocabulary (ColumnDef/TableDef enforcing type + nullability + length at construction, Nullability with no Default so omitting NULL/NOT NULL cannot compile silently), and query-expression AST vocabulary (Expr + operators, scoped to expressions per Decision 6's explicit deferral of the full statement grammar to Phase 4). Every newtype is repr(transparent) with a private field and a TryFrom constructor; no infallible From exists where an invariant could fail. CRATE-INDEX.toml declares the layer_order and per-crate dependency edges using each crate's own path-pinned Cargo.toml dependency, not workspace.dependencies, matching kanon's own convention and letting ARCHITECTURE/crate-index-conformance actually cross-check the graph. Every crate carries CRATE-SHAPE.toml with shape = unclassified per CRATE-SHAPE.md's registry contract, since a durable shape is a review decision this PR does not invent.
Resolved by kanon gate --tier full while validating this branch. Committing the lockfile matches the fleet convention (heurema commits its own) and pins the dependency graph deterministically for CI.
Pinax carried only gate-attestation.yml (trailer-only, no build fallback) and release-please.yml -- the sole path to landing non-docs code was a locally-produced Gate-Passed trailer. That gap was fine while pinax was docs-only; it became a real problem the moment #10 landed a four-crate workspace with no build CI to catch it. Adds ci.yml (fmt/check/clippy/nextest matrix, modelled on sphragis's job shape and heurema's --workspace/nextest split, since pinax is a workspace like heurema) and security.yml + osv-scanner.toml (cargo-deny/cargo-audit/osv-scanner, modelled on sphragis's pinned-binary cargo-audit install rather than heurema's source build, which couples the scanner's MSRV to the crate's). Both are informational until gate-attestation.yml migrates to the hybrid-gate pattern sphragis and heurema already run. Also lands the two drift items #10's own PR body flagged and deliberately left: .kanon-ci.toml now runs the real Rust gate stages its own comment named (cargo fmt/check/clippy/nextest, concurrency- capped per basanos CI.md) ahead of the existing kanon-lint stages, and drops the now-inapplicable disabled [verifier] override now that a real workspace exists to probe. release-please-config.json gains extra-files entries so a release bump updates the workspace version and every internal path-dependency pin that hardcodes it -- without them a release would bump lexis to a new patch while every crate that depends on it via path+version stayed pinned to the old one, breaking the build.
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
clippy::double_must_use fired on ColumnDef::new, ColumnDef::check_value, TableDef::new, and SqlType::check_comparable: each carried an explicit #[must_use] on top of a Result<_, LexisError> return, and Result is already #[must_use] at the type level under -D warnings. RUST.md's "must_use on every public Result-returning fn" rule is what produced these -- tracked as kanon#3473, unresolved. Until it lands, drop the redundant attribute; Result's type-level must_use already carries the guarantee the rule was reaching for. #[must_use] stays on builder methods and pure non-Result functions, which are correct and were never flagged.
…file deny.toml was copied from heurema (MPL-2.0) without adapting two entries to pinax's own posture: - licenses.allow was missing LicenseRef-PolyForm-Shield-1.0.0, pinax's own declared workspace licence (Cargo.toml). cargo-deny rejected all four crates. kanon's own deny.toml already carries this entry. - once_cell v1.21.4 is on the ban list (LazyLock stabilized in 1.80) but arrives transitively via lexis's dev-dependency chain proptest -> tempfile -> once_cell. The ban exists to keep our own code off once_cell, not to forbid every transitive dependency's choice, and tempfile's dependency graph isn't ours to control. `wrappers` scopes the exception to exactly that chain -- once_cell stays denied everywhere else -- rather than dropping the ban or dropping proptest, which the fleet testing standard requires for property coverage here.
kanon lint reported 15 warnings on this branch; the gate treats any warning as red. Fixes the 13 this PR introduced and 2 that predate it: - MANIFEST/maturity-description-mismatch x3: hypomnema, phylaxis, and pinax declare maturity = "scaffold" but carried production-tone descriptions. Prefixed each with "(scaffold)" per MANIFEST.md; also dropped "Sovereign" from pinax's description (WRITING/identity-fluff, kanon#2984 -- an identity claim, not a description of what the crate does). - TESTING/no-tests x4: hypomnema, phylaxis, and pinax are empty scaffolds by design at this phase (doc-comment-only lib.rs, zero behavior) -- suppressed via .kanon-lint-ignore, citing each crate's own exit-criteria as the drop condition, rather than writing a tautological test with nothing real to assert. lexis's lib.rs is a different case: it is NOT empty (41 tests), the rule only inspects lib.rs itself and a sibling tests/ dir and cannot see the schema.rs and types.rs submodule tests -- same false-positive shape as kanon's own existing TESTING/no-tests:crates/mnemosyne/src/lib.rs entry. - STORAGE/no-migration-checksum x2: verified against the rule's own implementation (has_ddl bare-matches CREATE/ALTER/DROP TABLE with no requirement that it sit near an execution call, unlike has_migration_runner's identifier-shape requirement). Both lexis files are pure type-vocabulary doc comments naming "CREATE TABLE" as a statement type the crate explicitly does not implement. Genuine false positive -- evidence posted to kanon#2975 rather than filing a duplicate; suppressed via .kanon-lint-ignore referencing that issue (not inline: the violation reports at line 1, which is the doc comment itself, so an inline marker would leak into rendered rustdoc). - TESTING/tautological-test x2: ast.rs's assert_eq! compared two textually-identical Expr::IsNull(column()) calls; named the two independently-constructed instances so the derived-PartialEq check they exercise isn't confused for a self-comparison. types.rs's prop_assert! wrapped a bare bool with no recognized verification macro in the body per the rule's heuristic; switched to prop_assert_eq!, matching the sibling property test one function up. - RUST/import-order x1: value.rs's test module put a crate-local use super::* before the external proptest import; std/external/crate is the required order. - WRITING/identity-fluff x1: covered above (pinax's description). - SHELL/unpinned-action x2 (pre-existing): gate-attestation.yml and release-please.yml called forkwright/.github's reusable workflows at the main branch ref, a mutable ref. Same defect class already fixed in hamma#91 -- pinned both to the current main commit SHA with a WHY and the refresh command. kanon lint . --all: 0 warnings, 0 errors (6 suppressed, all documented above).
Gate-Passed: kanon 0.12.0 +stages:fmt,check,clippy,nextest,lint sha:e6789841601ba770c40cf6a50f7ba3eedcc935a6
forkwright
pushed a commit
that referenced
this pull request
Aug 16, 2026
Lands ROADMAP.md Phase 01 ("pager + buffer pool + B-tree") on top of the
workspace + lexis scaffold: a checksummed page format, a copy-on-write
pager, an LRU-evicting buffer pool, and an integer-keyed B+tree, exposed
as pinax::Database.
Page format (Decision 2): PageSize validates the five locked sizes
(4096-65536), each page carries a trailing XxHash3-64 checksum
(xxhash-rust, pure Rust), max_local = usable_space - 35 per the copied
SQLite formula. Rows over max_local spill into an overflow page chain.
Pager (Decision 1): the meta/header lives in two checksummed 4 KiB slots
(page ids 0 and 1) at fixed file offsets independent of the configured
page size, so bootstrap never depends on a value it has not read yet.
Every mutation targets a freshly allocated page id (no page reachable
from the active meta is ever overwritten in place); commit fsyncs data,
writes the other meta slot with an incremented txn_id, fsyncs again, then
flips which slot is active. A crash at any point leaves the previously
committed slot intact, which is what makes crash safety possible without
a WAL, a Phase 02 deliverable.
Buffer pool: capacity-bounded LRU cache over the pager. Because every
write is copy-on-write, get only ever needs to clone a page out (never
checked out, never invalidated by an in-progress edit), so eviction has
no aliasing hazard and needs no pinning.
B+tree: slotted leaf/interior pages, path-copying insert/delete (root to
leaf gets a fresh id chain, propagated bottom-up through
apply_result_to_interior), rows encoded as lexis::Value tuples per
lexis's own stated exit criterion. Delete does not merge or rebalance
underflowing nodes, which is out of Phase 01's acceptance criteria and
tracked as follow-up rather than silently dropped.
Wires the real Rust gate into .kanon-ci.toml (fmt/check/clippy/nextest +
kanon lint), per that file's own comment naming this as the point to do
it, and updates README/CRATE-SHAPE status text to match.
Part of #10
forkwright
added a commit
that referenced
this pull request
Aug 16, 2026
## Summary
Lands ROADMAP.md Phase 01 ("pager + buffer pool + B-tree") on top of
#10's workspace + `lexis` scaffold. Every success criterion in that
phase gets a real test: open a file, CRUD rows by integer key, survive
crash-and-reopen, page-format checksums verified with corruption
detected, buffer pool handles a database larger than its capacity.
Base branch is #10's `feat/2-phase01-workspace-lexis`, not `main` — this
PR should merge (or be rebased) after #10.
## Page format (Decision 2)
- `PageSize` validates the five locked sizes (4096 / 8192 / 16384 /
32768 / 65536), default 4096.
- Every page — meta, leaf, interior, overflow — carries a trailing
8-byte XxHash3-64 checksum (`xxhash-rust`, pure Rust, matching Turso's
algorithm choice per the design-prior citation in PLAN.md).
- `max_local = usable_space - 35`, the SQLite formula PLAN.md names
verbatim. Rows over that threshold spill into an overflow page chain
(own encoding, not SQLite's varint format — Decision 1 says pinax owns
its on-disk encoding).
## Pager (Decision 1)
The meta/header lives in **two** checksummed 4 KiB slots (page ids 0 and
1) at fixed file offsets, independent of the database's configured page
size — bootstrap has to learn `page_size` from the meta page before it
can compute anything `page_size`-relative, so the meta region's own
layout can't depend on that value.
Every mutation is copy-on-write: nothing already reachable from the
active meta page is ever overwritten in place, every change targets a
freshly allocated page id. Commit fsyncs the data pages, writes the
*other* meta slot with an incremented `txn_id`, fsyncs again, then flips
which slot is active. A crash at any point leaves the
previously-committed slot intact — there's nothing to roll back because
nothing reachable ever changed. This is the mechanism the
crash-and-reopen test demonstrates, and it needs no WAL (that's Phase
02).
**Deliberate Phase 01 scope limit:** no freelist reclamation. A page
retired by a transaction's copy-on-write is still referenced by the
*currently active* meta until that transaction's own meta write lands,
so reusing its id before that point would corrupt the
still-authoritative pre-transaction state on a crash in between. Reusing
an id retired by an *earlier, already-committed* transaction is safe in
principle, but Phase 01's allocator is a pure bump counter that doesn't
attempt it — the database grows monotonically instead. None of Phase
01's acceptance criteria require space reclamation; this is real,
deferred scope, not a correctness gap.
## Buffer pool
Capacity-bounded LRU cache over the pager. Because every write is
copy-on-write, `get` only ever needs to *clone* a page's bytes out —
nothing is ever checked out or held mutably against the pool, so
eviction has no aliasing hazard and needs no pinning bookkeeping. A page
evicted mid-operation is safe to flush early for the same reason a crash
mid-operation is safe: it isn't reachable from the active meta yet
either way.
The "buffer pool handles a database larger than RAM" test models RAM as
pool capacity (8 pages), inserts 4,000 rows to produce a tree many times
that size on disk, and reads every row back correctly after
eviction-and-reload cycles.
## B+tree
Slotted leaf and interior pages (SQLite-shape layout per Decision 1, own
encoding). Insert/delete path-copy: the whole root-to-leaf path gets a
fresh id chain, propagated bottom-up (`apply_result_to_interior`), with
splits handled at both leaf and interior levels. Rows are encoded as
`lexis::Value` tuples — `lexis`'s own `Cargo.toml` names this as its
Phase 01 exit criterion ("Phase 1 pager adopts `lexis::Value` for
on-disk row encoding").
**Deliberate Phase 01 scope limit:** delete does not merge/rebalance
underflowing nodes. `delete`'s interior-propagation step only ever
repoints an existing child pointer, it never removes a separator key, so
an ancestor's key count is monotonically non-decreasing — the tree stays
correct (an empty leaf is a valid leaf) but not space-optimal under a
delete-heavy workload. None of Phase 01's acceptance criteria require
this; general merge-on-underflow is real follow-up scope for a later
phase, not silently dropped.
## Testing
- `tests/phase01_acceptance.rs`: one test per ROADMAP.md Phase 01
criterion, named and cited against the exact wording. The corruption
test flips a byte on disk and asserts a typed `Corruption` error — the
negative case, not just the intact-page happy path. The crash test drops
a `Database` mid-stream with no explicit shutdown and reopens.
- Every module carries colocated `#[cfg(test)] mod tests` for its own
internals — codec bounds-checking, checksum round-trip and
flip-detection, meta-slot fallback on corruption (a second, independent
demonstration of crash-safety: corrupt the *active* meta slot directly
and confirm `Pager::open` falls back to the still-valid prior slot),
pager crash-before-commit, B+tree
CRUD/split/overflow/negative-key/reopen, buffer-pool
eviction-flushes-dirty and corrupted-page-surfaces-on-read.
- Every public fallible constructor and every error variant used in
Phase 01 has a test that produces it.
## Scope decisions worth flagging
- **Row payload is `lexis::Value`, not a typed/schema-checked row.**
Per-column type checking against a `TableDef` is Phase 4/5 territory
once `CREATE TABLE` exists; this crate stores a self-describing value
tuple.
- **Single implicit B+tree per file**, matching Decision 1's "one file
per database" literally. A named-table catalog is Phase 04's `CREATE
TABLE` layered on top of this same engine.
- **Synchronous, not async.** Decision 12 locks async-first for the
eventual pager surface, but that's phrased around WAL/tokio machinery
that doesn't exist yet (Phase 02+), the scaffolded `pinax` crate carried
no tokio dependency, and no Phase 01 acceptance criterion mentions
async. Flagging as a spec gap rather than silently deciding — Phase 02
is the natural point to revisit this once the WAL trait lands.
- **`.kanon-ci.toml` updated** to the real Rust gate
(fmt/check/clippy/nextest + kanon lint), replacing the docs-phase stage
list — that file's own comment named this as the point to do it, since
code has now landed.
## Verification
metis is under heavy concurrent load from other sessions (1-min load
20-35 against an 8-thread box, `vgate --status` showing waiters queued
30-50+ minutes) for the duration of this work, so `vgate` never admitted
a local `cargo check`/`clippy`/`nextest` run despite being queued for
the whole session. `cargo fmt --all` (not gated) ran clean, and `cargo
generate-lockfile` / `cargo tree` / `cargo metadata` (also not gated)
confirm the dependency graph and every manifest resolve correctly with
no version conflicts. The code has had an extensive manual review pass —
several real issues (an `as` cast, a handful of
`PermanentError`/`FatalError` → `PinaxError` conversion sites needing
`#[snafu(transparent)]`, a meta-slot encoding bug that zeroed two magic
bytes, an inaccurate doc claim about root-collapse actually being live)
were caught and fixed that way — but a real `cargo
check`/`clippy`/`nextest` run has **not** been observed to pass locally
or in CI as of this PR opening.
**CI will not run on this PR yet, separately from the load issue
above:** `gate-attestation.yml` triggers only `on: pull_request:
branches: [main]`, and this PR's base is #10's branch, not `main` (per
this task's own instruction to build on the scaffold rather than race
it). No check will appear here until either #10 merges (this PR's base
then follows main) or this PR is retargeted. See `blocked_on` in the
task handoff for the full `Gate-Passed` trailer status.
---------
Co-authored-by: forkwright <cody@forkwright.com>
forkwright
pushed a commit
that referenced
this pull request
Aug 16, 2026
🤖 I have created a release *beep* *boop* --- ## [0.0.3](v0.0.2...v0.0.3) (2026-08-16) ### Features * **pinax:** implement Phase 01 pager, buffer pool, and B+tree ([#11](#11)) ([8142a84](8142a84)) * **workspace:** stand up the pinax workspace and implement lexis ([#10](#10)) ([234be1a](234be1a)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stands up the four-crate pinax workspace per Decision 14 and implements
lexisfor real per Decision 5. This is the foundation the rest of the phase ladder builds on — deliberately narrow, nothing beyond workspace +lexis.Cargo.tomlwith[workspace.package]/[workspace.dependencies]/[workspace.lints], locked dependency graphlexis -> hypomnema -> phylaxis -> pinax.deny.toml(copied from heurema's template),clippy.toml, minimalrustfmt.toml(edition = "2024"only),.config/nextest.toml..gitattributesalready carried the markdown whitespace carve-out — no change needed there.hypomnema/phylaxis/pinax: conforming empty library crates —lib.rswith#![deny(missing_docs)], a module doc naming what they'll own and when (Decision 14), and nothing else. TheirCargo.tomls declare the real path-pinned dependency edges the graph requires, even though the crate bodies don't use them yet.lexis: implemented. The strict six-type system (SqlType:INTEGER | REAL | TEXT | BLOB | BOOLEAN | DATETIME,#[non_exhaustive], noNUMERIC/DECIMAL/DATE/TIME/JSON), theValueruntime enum (NULLas a first-class variant for three-valued logic), schema DDL vocabulary (ColumnDef/TableDefwithNullabilitycarrying noDefault), and query-expression AST vocabulary (Expr+ operators).CRATE-INDEX.toml:layer_order+ per-crate entries, using each crate's own path-pinnedCargo.tomldependency (matching kanon's own convention, not[workspace.dependencies]) soARCHITECTURE/crate-index-conformanceactually cross-checks the graph instead of finding nothing to check.CRATE-SHAPE.toml: one per crate,shape = "unclassified"per CRATE-SHAPE.md's own registry contract — a durable shape is a review decision, not one this PR invents.Type discipline (the point of this crate)
Every newtype (
RealValue,DateTimeValue,ColumnName,TableName,TextMaxLen) is#[repr(transparent)]with a private field and aTryFromconstructor; no infallibleFromexists where an invariant could fail.RealValue::try_fromrejects NaN at the boundary Decision 5 names ("computed values may be NaN; insert-bind rejects it").ColumnDef::check_valueis the one place Decision 5's "values inserted into a column must be of that type or NULL... implicit conversions are rejected" actually lives, so every future caller (pager row encoder, executor INSERT/UPDATE path) enforces it identically instead of re-implementing it.Nullabilitydeliberately has noDefault— acompile_faildoctest proves omitting the NULL/NOT NULL annotation cannot compile silently, matching Decision 5's "omitting the annotation is a parse error."Scope decisions worth flagging
SqlType::check_comparableis a pure type-level compatibility check (INTEGER↔REAL permitted, same-type permitted, everything else a type error) — it does not perform value comparison. Decision 5's full comparison semantics (magnitude ordering, the f64-mantissa-exactness rule for INTEGER↔REAL, three-valued NULL propagation) is executor behavior; Decision 14 assigns execution to the pinax facade's Phase 4/5 planner and executor, not this crate's vocabulary. This also sidesteps a real gap: there is no losslessi64 -> f64conversion instd(noFrom/TryFrom), so implementing the value-level comparison here would have forced either anascast (against this task's explicit "noascasts" bar) or an invented workaround — the correct fix is that this isn't lexis's job.ColumnName/TableNamevalidate non-emptiness only. Decision 5/6 don't specify SQL identifier charset, max length, quoting, or reserved-word handling — that's Phase 4 (parser) territory. Inventing a charset or length cap here would be a decision this crate has no authority to make.Expr+UnaryOperator/BinaryOperatorcover the query-expression vocabulary Decision 14 assigns to lexis. The full statement grammar (SELECT/INSERT/UPDATE/DELETEwith joins, subqueries, window functions) is explicitly Phase 4 per Decision 6 ("SELECT alone has precedence tables, correlated subqueries, lateral joins, window functions" is named there as why a combinator parser doesn't scale) and ROADMAP.md.TableDef/ColumnDefadd two invariants beyond Decision 5's literal text: zero columns and duplicate column names are both rejected. Neither is an invented pinax-specific rule — no relational model admits either, so treating them as validation-boundary invariants is a direct reading of "parse, don't validate," not a new decision..kanon-ci.tomlandrelease-please-config.jsonare untouched..kanon-ci.toml's own comment says to replace the docs-phase pipeline stages "when code lands" — that's real, caused by this PR, but wiring the forge's remote CI pipeline is a different concern than workspace + lexis and wasn't in this task's scope. Flagging as a deliberate follow-up rather than silently leaving it, per the fleet's drift-handling convention.Testing
Every public fallible constructor and every
LexisErrorvariant has a test that produces it. Proptest coverage on the two real universally-quantified invariants:RealValue::try_fromaccepts iff not NaN (across the wholef64domain, bit-pattern round-trip on the accepted half) andSqlType::check_comparableis symmetric (all 36 ordered type pairs). Nounwrap/expectin library code (#![deny(clippy::unwrap_used, clippy::expect_used)]at the crate root with a#[cfg_attr(test, allow(...))]escape); noascasts anywhere; no indexing —TableDef::check_value's length-bound check usesu32::try_from(len).is_ok_and(...)specifically to avoid one.Verification
metis is loaded (concurrent gate/build sessions on the same 8 threads);
cargo fmt --all -- --checkran clean locally. A fullkanon gate --tier full --stamprun was launched for theGate-Passedtrailer this repo's branch protection requires (gate-attestation, trailer-only, no build fallback) — see the PR's commit history for whether it landed. If it didn't complete before this PR opened, the branch is intentionally left without a fabricated trailer; a follow-up push carries a real one once a local gate run completes or CI surfaces a result.