feat: on-disk encryption (XChaCha20-Poly1305, envelope keys, O(1) rotation) - #85
Merged
Conversation
…ncryptionKey, EncryptionNotSupported, NoFreeKeySlot, LastKeySlot, fatal DecryptionFailed)
- Remove derive(Debug) from Kek: Zeroizing's derived Debug delegates to the inner array without redaction, so it would print raw key bytes. - Make the intermediate KDF output buffer Zeroizing and move it into Kek, so no un-wiped copy of derived key material lingers on any path.
Add seal_detached/open_detached AEAD helpers (reused by Task 1.5 PageCipher) and the public wrap_dek/unwrap_dek API. Detached mode keeps wrapped-DEK output the same 32-byte length as plaintext with a separate 16-byte Poly1305 tag, matching the fixed key-slot layout. AAD binds slot metadata so slot parameters cannot be silently tampered. Any auth failure returns CryptoError::Auth with no partial plaintext leakage; intermediate key bytes kept in Zeroizing buffers. Five new tests: round-trip, wrong KEK, tampered tag, tampered ciphertext, wrong AAD — all use matches!/is_err() to avoid requiring Dek: Debug (by design).
open_detached returned a plain Vec<u8> holding the decrypted DEK; the copy into a Zeroizing array in unwrap_dek wiped only the copy, leaving the original heap buffer returned to the allocator un-scrubbed. Return Zeroizing<Vec<u8>> so the plaintext is wiped on drop. Also correct the open_detached comment: XChaCha20-Poly1305 is verify-then-decrypt (tag checked before keystream applied), so on auth failure the buffer still holds ciphertext and no plaintext is written — the previous "scrubs on failure" claim was wrong. Move the chacha20poly1305 imports to the top-of-file import group.
…gion Convert superblock.rs to a module directory and add superblock/crypto_header.rs with the KeySlot + CryptoHeader codec. On-disk layout occupies bytes 324–1356 in the superblock's reserved region (well inside CHECKSUM_OFFSET 8184): an 8-byte preamble (algorithm u8 + stride u32 + 3 reserved bytes) followed by 8 × 128-byte key slots. Algorithm byte 0 → plaintext DB (deserialize returns None); algorithm 1 → XChaCha20-Poly1305. Also derives PartialEq + Eq on Argon2Params, required by KeySlot's own PartialEq derive.
Adds `pub encryption: Option<CryptoHeader>` to `Superblock`. For encrypted DBs, `serialize_encrypted` seals all sensitive fields (root pointers, total_pages, next_handle, freemap_depth, named_roots) under the DEK into a nonce||tag||len||ct blob at SEALED_BODY_OFFSET (1356), leaving those byte ranges zero in the on-disk page. `deserialize` reads the crypto-header first; for encrypted DBs it returns the struct with zero sensitive fields and populated `encryption` field, deferring fill to `decrypt_body` once the DEK is known. Plaintext DBs use the unchanged plaintext path. Also adds `sb_identity_aad` (AAD = magic|format_version|txn_counter| superblock_count, preventing body transplant between DBs) and updates all Superblock struct literals (including commit.rs and recovery_tests.rs) with `encryption: None`.
The page checksum is non-cryptographic XXH3, so a forged page can carry an out-of-range ct_len whose slice runs past CHECKSUM_OFFSET and panics before open_body's AEAD auth can reject it. Guard the slice: an out-of-range length now returns CryptoError::Auth (the same undecryptable-body error as a failed auth), so a corrupt sealed body is recoverable rather than a process crash. Also: assert the membership-index and freemap_depth byte ranges (312..324) are zero in the encrypted page (sealed-only, must not leak in cleartext), and add a debug_assert_eq on load_body's input length to document the BODY_LEN invariant. New test forged_ct_len_returns_err_not_panic regresses the panic.
…rypt body, gate MAJOR=2
…d-body decrypt The open_existing path reconstructed the winner's buffer index using txn_counter % superblock_count, which matches the commit write-slot rule but is inverted for the create_new seeding rule (slot i gets counter N-1-i). For a freshly created, never-committed encrypted DB the winner (counter N-1) sits at page 0, but (N-1) % N = N-1 points at the loser slot, so decrypt_body built its AAD from the winner's txn_counter but read the sealed body from the wrong buffer -- AAD mismatch -- CryptoError::Auth -- correct key wrongly fails. Fix: replace slot_idx/raw derivation with filter_map + max_by_key over (deserialized_superblock, &buffer) pairs so raw is provably the same buffer the winner was deserialized from, regardless of seeding vs commit ordering. Superblock::select() (which discards the buffer) is now #[cfg(test)] only. Also fix the UnsupportedFormatVersion expected field: report format_version_encrypted() for an encrypted DB's MAJOR mismatch instead of the plaintext FORMAT_VERSION, which was misleading for a MAJOR=2 file. Regression test: open_encrypted_db_with_no_commits_uses_correct_key confirms a correct key opens a never-committed encrypted DB (the exact broken scenario).
…al invariants Add two unit tests (encrypted_manager_holds_session_cipher, plaintext_manager_has_no_cipher) confirming that TransactionManager retains Option<PageCipher> correctly after create_new: Some for an encrypted DB, None for a plaintext DB. DEK zeroization on drop is automatic via Zeroizing<[u8; 32]> inside PageCipher; safe Rust cannot observe post-drop memory, so the tests assert the structural invariant rather than attempting to read freed memory. Task 2.4 already completed all implementation (cipher field, constructor wiring, call-site updates). This task is verification + targeted tests.
…192 plaintext) - Backing::Memory switches from Vec<[u8; PAGE_SIZE]> to flat Vec<u8> addressed by page_id * stride, enabling variable on-disk unit sizes. - PageIo gains a stride field (default PAGE_SIZE). New set_stride() / stride() accessors let the engine switch to ENC_PAGE_SIZE (8232) after reading the plaintext page-0 bootstrap header on an encrypted open. set_stride re-seeds the I51 cached page count in the new unit size. - New read_page_unit() / write_page_unit() move stride-byte blobs. Crypto-agnostic: for encrypted DBs the blob is sealed ct|tag|nonce; PageCache handles seal/open (Task 3.3). For plaintext stride==PAGE_SIZE and the blob is the page image itself. - read_page() / write_page() become thin wrappers over the unit functions (debug_assert-guarded to catch any accidental encrypted-path call). - set_page_count() now sets file length to n*stride rather than n*PAGE_SIZE. - 6 new stride_tests cover: 8232 round-trip, default stride, wrong-length error, set_stride reseeds count, set_page_count uses stride, plaintext wrappers unchanged.
…port Widens the spillway slot from the fixed PAGE_SIZE (8192) plaintext payload to a runtime-configurable payload_size, enabling it to carry the 8232-byte ENC_PAGE_SIZE sealed blob for encrypted DBs. Changes: - Spillway::open_file / open_memory gain a payload_size: usize param (PAGE_SIZE for plaintext, ENC_PAGE_SIZE for encrypted). - SLOT_SIZE const retained as the plaintext-default for existing test sizing; runtime slot arithmetic now uses SLOT_HEADER_SIZE + payload_size. - spill() accepts &[u8] instead of &[u8; PAGE_SIZE]; rehydrate() returns Vec<u8> instead of [u8; PAGE_SIZE] — crypto-agnostic: sealed blob stored and returned verbatim. - logical_bytes() and SpillwayFull cap charge against payload_size. - page_cache.rs: pass PAGE_SIZE at open_file/open_memory; convert rehydrate Vec<u8> back to [u8; PAGE_SIZE] via try_into (plaintext path); .as_ref() on spill call to coerce Box<[u8; N]> to &[u8]. - New tests: wide_slot_round_trips_sealed_blob and wide_slot_checksum_catches_tampered_payload verify 8232-byte round trips and checksum detection at ENC_PAGE_SIZE.
…sk 3.3) Encrypt data pages at flush, decrypt at cold load, and make an encrypted database use the ENC_PAGE_SIZE (8232) stride for every page — superblock slots included — from creation. PageCache: - Add Option<PageCipher>; entries always hold plaintext, the main file and spillway hold ciphertext. - write_sealed() seals at flush (Phase 1a dirty-write and 1b drain) when a cipher is present, else writes the plaintext unit directly. - load_page cold path reads the on-disk unit into a stack [u8; ENC_PAGE_SIZE] (zero heap alloc via read_page_unit_into), opens it through the cipher, then verifies the checksum on the recovered plaintext. A CryptoError maps to the fatal DecryptionFailed. page_io: add read_page_unit_into (zero-alloc cold-load read); drop the now-dead allow(dead_code) on stride()/set_stride(). Uniform-stride layout: - create_new sets the 8232 stride BEFORE writing the initial superblock slots (each 8192-byte image zero-padded into an 8232-byte unit), so a fresh or never-committed encrypted file is a clean multiple of the stride and page_count == total_pages at open. - commit.rs writes the superblock into a zero-padded 8232-byte unit for encrypted DBs (write_page would panic on the stride assert). - open_existing anchors on page 0 (always at offset 0), learns the stride from its cleartext crypto-header, then reads all slots at that stride before the file-size validation. A torn slot 0 (no slot deserializes at the default stride) speculatively retries the scan at the encrypted stride so R4 sibling recovery still works for encrypted DBs. - FileSizeMismatch reports stride-correct byte counts. crypto: derive Clone on PageCipher (cache and session manager each hold an independent zeroizing DEK copy); remove the module-level allow(dead_code). Tests: encrypted page seal/open round-trip, tamper -> DecryptionFailed, never-committed N=4 reopen, multi-page value round-trip, and torn-slot-0 sibling recovery (confirmed to fail when the stride fallback is disabled).
Closes the plaintext-spill leak flagged in the Task 3.3 report. - ensure_spillway: select payload_size=ENC_PAGE_SIZE when a cipher is present so the spillway slot is sized for the sealed blob from the start. - maybe_evict Phase B: seal the plaintext page once (via PageCipher::seal) before handing the ciphertext blob to Spillway::spill. The spillway now NEVER holds plaintext for an encrypted DB. - flush Phase 1b drain: the spillway slot already holds the sealed blob; write it verbatim to the main file via io.write_page_unit (no second seal, no new nonce). Open the blob to obtain plaintext for the cache re-insertion that follows drain. - load_page spillway branch (rehydrate path): open the sealed blob back to plaintext before re-inserting the still-dirty page into the cache. - Plaintext DBs: payload_size stays PAGE_SIZE; no cipher branch taken; byte-identical to the pre-3.4 behaviour. Five new tests in page_cache::tests: encrypted_spill_stores_ciphertext_not_plaintext encrypted_spill_drain_and_cold_read_round_trips encrypted_rehydrate_returns_correct_plaintext encrypted_spill_full_round_trip_multiple_pages plaintext_spill_drain_unchanged
…ions - Make Options::encryption_key pub (was pub(crate)); callers no longer need to name crate::crypto::Key to use it - Add Options::argon2_params: Option<Argon2Params> field and builder for caller-supplied Argon2id cost parameters on passphrase-key creates - Rename Options::with_encryption_key -> encryption_key to match the field-name-as-builder convention used by all other Options setters - Thread argon2_params through create_new -> build_create_cipher so caller-supplied params actually override the OWASP default for Key::Passphrase creates - Add tests/public_key_api.rs proving Key/Argon2Params are reachable from the public surface without naming crate::crypto internals - Add options_encryption_tests unit test in src/lib.rs
Exposes chisel::Key to Python via the pyo3 open() binding. bytes → Key::Raw(Zeroizing::new(...)), str → Key::Passphrase(Zeroizing::new(...)), anything else → TypeError. Key material is wrapped in Zeroizing immediately under the GIL before py.detach. Also adds the four encryption exception classes (Task 4.5 scope folded in so the test file runs end-to-end): NoEncryptionKeyError, InvalidEncryptionKeyError, EncryptionNotSupportedError (OperationalError), DecryptionFailedError (FatalError). Stubs and __init__.py updated.
Add group 12 to test_exception_contract.py pinning the four encryption exception classes (concrete class + tier base). NoEncryptionKeyError, InvalidEncryptionKeyError, EncryptionNotSupportedError triggered end-to-end; DecryptionFailedError covered by FatalError hierarchy check only (per-page MAC tampering not reachable from pure Python, matching the existing fatal-arm precedent). Drop unused derive feature from the zeroize dep (only Zeroizing::new used). Comment the intentional bare open() in test_encryption.py.
Adds TransactionManager::rewrite_crypto_header (src/transaction/keys.rs), which persists a modified CryptoHeader via the ordinary A/B superblock rotation — no data pages touched, DEK unchanged. Guards against poisoned manager, active transaction, and plaintext DB. Uses the same ENC_PAGE_SIZE zero-padding + write_page_unit path that commit.rs uses for encrypted DBs, so the on-disk stride invariant is maintained. Seven unit tests cover all guards, in-memory state promotion, root preservation, slot alternation, and full on-disk reopen durability.
O(1) credential rotation via DEK re-wrap: add_key stages a second wrapped copy of the existing DEK into a free key slot; rotate_key does the same then clears the old slot in the same single atomic superblock rewrite so there is never a window where zero credentials unlock the database. Remove #[allow(dead_code)] from rewrite_crypto_header (now has callers). 8 integration tests in tests/encryption_keys.rs cover both methods across plaintext DB, wrong credential, full slot table, and round-trip data readability.
…ror/LastKeySlotError - errors.rs: create_exception! for NoFreeKeySlotError and LastKeySlotError (both OperationalError); add to_py_err arms for NoFreeKeySlot/LastKeySlot variants; register both in module init; update hierarchy comment. - db.rs: add module-level py_key() helper (bytes→Key::Raw, str→Key::Passphrase, else TypeError); add three #[pymethods]: add_key(existing, new), rotate_key(old, new), remove_key(key) — each coerces via py_key then calls with_inner_mut_io. - chisel/__init__.py: import + __all__ entries for both new error classes. - chisel/chisel.pyi: stubs for the two new error classes and three new Chisel methods. - python/tests/test_encryption_keys.py: new test file — add_key (both keys open, passphrase alongside raw), rotate_key (old revoked, two-slot), remove_key (revokes slot, other key works), LastKeySlotError, NoFreeKeySlotError, bad key type TypeError. - python/tests/test_exception_contract.py: section 13 — end-to-end triggers for both new exception classes, tier (OperationalError) hierarchy pin.
…ion test Adds FORMAT_MINOR_VERSION_ENCRYPTED (0) and ENCRYPTED_FORMAT_VERSION (pack(2,0)) to page.rs alongside the existing FORMAT_MAJOR_VERSION_ENCRYPTED. Exports both new constants from lib.rs. Adds encrypted_major_version_is_rejected_by_plaintext_binary in a new #[cfg(test)] mod in recovery.rs: writes a superblock with format_version=2 (no crypto header), opens without a key, and asserts UnsupportedFormatVersion is returned — proving the format gate hard-rejects MAJOR=2 files on an encryption-unaware path, exactly as the on-disk-encryption spec §7/§8 require.
… pack(2,0) The first pass left two representations of the encrypted format version that disagreed on the MINOR byte: format_version_encrypted() returned pack(2, 1) while the new ENCRYPTED_FORMAT_VERSION const was pack(2, 0). Collapse to one: - format_version_encrypted() now returns ENCRYPTED_FORMAT_VERSION (pack(2, 0)). A new MAJOR series correctly starts its MINOR count at 0. Create, commit, open-gate, and the superblock-body AAD all route through this one function, so there is exactly one on-disk value. - Scope FORMAT_MINOR_VERSION_ENCRYPTED and ENCRYPTED_FORMAT_VERSION to pub(crate); revert the lib.rs re-export to the pre-task public surface. - Gate-rejection test now asserts the exact stamped value (0x0002_0000).
…rate) Items added by the on-disk encryption feature (CryptoHeader, KeySlot, ALGO_XCHACHA20POLY1305, CRYPTO_HEADER_OFFSET, KEY_SLOT_COUNT, KEY_SLOT_SIZE, FORMAT_MAJOR_VERSION_ENCRYPTED, format_version_encrypted, PageCipher, CryptoError, KdfId, derive_kek, wrap_dek, unwrap_dek, NONCE_LEN, SALT_LEN, DEK_LEN) were incorrectly exported on the crate's public surface. These items are internal implementation details scoped inside pub(crate) modules (crypto, superblock); they do not need explicit re-exports in lib.rs. Removing the pub use lines from lib.rs confines them to pub(crate) access, which is all in-crate callers need. Pre-existing public items left unchanged: SlotDefect, SuperblockDefect, DEFAULT_SUPERBLOCK_COUNT, MAX_SUPERBLOCKS, MIN_SUPERBLOCKS, NAMED_ROOT_COUNT, NAMED_ROOT_NAME_LEN, format_major (all public before this branch). Public API additions: Key and Argon2Params (callers must supply these to open encrypted databases).
…rruption Four .expect() calls in page_cache.rs on try_into() during spillway rehydration bypassed the I1 poison model with a panic on internal corruption (wrong-length blob). Replaced with .map_err(|_| DecryptionFailed) so the typed error propagates and the poison wrapper fires as intended. wrap_into() in superblock/crypto_header.rs panicked via .expect() if derive_kek returned an error (e.g. caller passes a zero-length key). Changed the function signature to return Result<(), CryptoError> and propagate the error. Callers in transaction/keys.rs map the CryptoError to InvalidEncryptionKey; test callers use .expect() (panics are fine in tests). Also replaced the bare state == 1 literal in is_active() with the new KEY_SLOT_ACTIVE named constant for consistency.
- PageCipher doc comment: clarified that the AEAD cipher is derived from the DEK on each seal/open call, not constructed once and reused. - error.rs encryption_error_classification test: "all four" -> "all six" (the loop covers six variants, not four). - ARCHITECTURE.md envelope description: corrected prefix layout from "1-byte algorithm id and 4-byte stride field" (implied 5-byte prefix, slots at byte 329) to the actual 8-byte prefix (1-byte algorithm id, 4-byte stride, 3 reserved bytes; key-slot table begins at byte 332).
…) scoping tests/encryption_create.rs referenced crypto/superblock/format internals (CryptoHeader, KeySlot, derive_kek, unwrap_dek, KdfId, ALGO_XCHACHA20POLY1305, CRYPTO_HEADER_OFFSET, KEY_SLOT_COUNT, KEY_SLOT_SIZE, FORMAT_MAJOR_VERSION_ENCRYPTED) that are now pub(crate). Moved the whole file to an inline #[cfg(test)] module at src/transaction/create_tests.rs, rewriting chisel::X imports to crate:: paths. Every assertion is preserved (create-artifact major-version, slot-0 population, named-root cleartext-leak check, wrap/unwrap round-trip). Old file deleted. Kek::from_bytes is now only reachable from tests (derive_kek builds Kek directly), so it is gated with #[cfg(test)] to avoid dead-code in release builds. derive_kek now rejects empty key material with BadKeyLength (it previously documented this but did not enforce it — HKDF/Argon2id silently accept a zero-length ikm). Refusing an empty caller-supplied key at this trust boundary.
…er fsync poison - Argon2id known-answer test in src/crypto/mod.rs: pins the exact 32-byte output of the derive_kek Argon2id path (cheap params) as a golden regression value, so a silent KDF config change (algorithm/version/params/output length) is caught. Documented as a golden pin (the RFC 9106 reference vectors use secret+AD parameters the derive_kek API does not expose). - In-memory encrypted round-trip in tests/encryption_roundtrip.rs: opens an encrypted in-memory DB via open_in_memory_with_options, writes and reads two values, updates one, verifies the other is unchanged. Plus a plaintext in-memory baseline so a regression in the in-memory path is distinguishable from an encryption-specific failure. Public API only. - rewrite_crypto_header fsync-failure poison test in src/transaction/keys.rs: arms Fault::FailFsync on the rotation path (via add_key) and asserts the manager is poisoned and rejects further ops with Poisoned, confirming the I1 poison-on-fsync-failure wiring covers key rotation, not just data commits. Plus a test that add_key with a zero-length raw key returns InvalidEncryptionKey rather than panicking.
Resolve the sole conflict in src/page_io.rs: main's commenting pass (#83/#84) corrected the fsync doc-comment from "twice" to "three times per commit" (I28 pre-drain flush + data pages + superblock), while this branch renamed the Memory backing field pages -> bytes. Kept both: the corrected "three times" comment (still accurate under encryption — the per-page seal changes the unit size, not the fsync count) and a match pattern valid for the renamed field (`Backing::Memory { .. }`). The other three shared files (python/src/errors.rs, src/defrag.rs, src/page_cache.rs) auto-merged cleanly. Verified on the merged tree: 680 tests across 32 suites pass, clippy clean on lib + chisel-py.
🚦 Bench results: PR vs main✅ No regressions detected
Per-scenario detail (4 metrics × cells)document-store
mutation-log
ycsb-a
ycsb-b
|
The CI `fmt` gate (cargo fmt --check) never ran on this branch — CI only triggers on PRs to main, and the branch was CONFLICTING until now, so no run was ever produced. Several encryption files carried unformatted code (hex-array literals, wrapped call args, struct literals). Pure rustfmt reflow; no logic change — full suite still 680 green.
The encryption feature pulled RustCrypto crates whose latest releases moved
to `edition2024` (Rust 1.85+), which broke the msrv CI job (builds -p chisel
on 1.82 — the documented library floor). Two edition-2021 floor pins, using
the same I61 tilde-pin mechanism already applied to clap in bench:
- zeroize "~1.8": 1.9.0 + its zeroize_derive macro are edition2024. We use
only Zeroizing<T> (no #[derive(Zeroize)]), so the `derive` feature is
dropped, removing zeroize_derive from the tree entirely.
- base64ct "~1.6": a hard transitive dep of argon2 0.5.3; 1.7+ is
edition2024. We never emit PHC hash strings (we call the raw
hash_password_into KDF), so the pinned version's code path is unused.
Verified by reproducing the exact CI job locally: `cargo +1.82 build
-p chisel` now Finishes. Full suite 680 green on stable; clippy + fmt clean.
This was referenced Jul 1, 2026
Xof
added a commit
that referenced
this pull request
Jul 1, 2026
…#87) An adversarial commenting pass over the on-disk encryption code (merged in #85). Comments only — no behavior change; full test suite, clippy, and fmt stay green. Wrong facts: - crypto/mod.rs: derive_kek's `# Errors` said BadKeyLength fires only for `kdf == Hkdf` with empty raw bytes; the empty-key check is unconditional (before the KDF match) and covers Raw and Passphrase. - superblock/mod.rs (x2): sealed-superblock comments said sensitive scalars 16..52 are zeroed, but page_size is cleartext at 48..52 — the zeroed range is 16..48. - python/src/db.rs: key coercion claimed an empty/bad raw key raises BadKeyLength via to_py_err; every CryptoError maps to InvalidEncryptionKey -> InvalidEncryptionKeyError (there is no BadKeyLength Python error). - transaction/recovery.rs: "Slot 0 was written last" — it is written first (i=0, highest counter superblock_count-1). - transaction/keys.rs (test): comment said txn_counter=3 after fresh_encrypted; it is 2. Stale: - page_io.rs: header still said "Two fsyncs per commit"; it is three (I28 pre-drain + data + superblock) — the header #84 missed. - transaction/commit.rs + lifecycle.rs: CommitCtx described as "ten pieces" of state; the cipher + crypto_header fields make it twelve. - python/src/errors.rs: exception-hierarchy comment omitted DecryptionFailedError from the FatalError tier. Three further candidates could not be adversarially verified (transient rate limit) and were left as-is rather than rewritten on an unverified claim.
Xof
added a commit
that referenced
this pull request
Jul 1, 2026
Constructive commenting pass (/comment-run) over the on-disk encryption surface merged in #85 — the complement to the wrong-comment fixes in #87. Additive only: ADD/EXTEND, no existing comment text deleted or rewritten. Comments only; full test suite, clippy, and fmt stay green. Each captures a non-obvious "we do X because Y" a first-time reader could not infer from the code: - crypto/mod.rs: Argon2id's Version::V0x13 and 32-byte output are pinned to the on-disk format (like KEK_INFO for HKDF) — changing either silently breaks unwrap of every existing Argon2id slot. - superblock/mod.rs: the four bootstrap fields must stay cleartext because they ARE the AAD, and slot selection + AAD derivation run before any DEK is available (a sealed body would be a chicken-and-egg deadlock). - page_cache.rs: the encrypted cold-load DecryptionFailed is fatal/poisoning, a peer of ChecksumMismatch, not a retryable operational error. - page_io.rs: set_stride swallows a seek error to 0 because its signature is infallible (bootstrap call site) and 0 makes reads fail closed. - spillway.rs: spill writes are deliberately not fsynced — content never crosses a transaction boundary, so durability would be wasted I/O. The pass found 0 wrong comments (fixed in #87) and 0 bugs. Six of the eleven modules reviewed needed nothing added — the encryption code was already well-commented from its per-task review gates.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds opt-in, client-supplied authenticated on-disk encryption. A client supplies a key when opening a database (
Options::encryption_key); every byte Chisel then writes is encrypted and integrity-protected. Without a key, databases are plaintext exactly as before — byte-for-byte identical, zero regression.Design
AAD = page_id(anti-relocation). Authenticated (not merely confidential) so tampering is cryptographically detected. Random nonces are crash-safe under shadow paging; a deterministic(page_id, counter)nonce would risk keystream reuse when a crashed transaction returns page_ids to the freemap while the durable counter has not advanced.PageCacheowns aPageCipher;page_iois stride-aware but crypto-agnostic). Encrypted DBs use this uniform stride from birth, including superblock slots; open bootstraps by reading page 0 at offset 0, learning the stride from its plaintext crypto-header, then reading the rest.named_roots), and the spillway (seal-once on evict, verbatim ciphertext copy on drain — never plaintext).add_key/rotate_key/remove_key. Rotation stages the new key before revoking the old (no zero-key window, even across a crash);remove_keyrefuses the last active slot (brick prevention). Persisted via an atomic metadata-only superblock commit.Design spec:
docs/specs/2026-06-29-on-disk-encryption-design.md. Decision + rejected alternatives recorded as ADR-15.What changed
src/crypto/(cipher, KDF, DEK wrap/unwrap, zeroizing key types),src/superblock/crypto_header.rs(key-slot table + sealed body),src/transaction/keys.rs(rotation + the metadata-only commit).page_io/page_cache/spillwaymade stride-aware with seal-on-write / open-on-read.Key,Argon2Params,Options::encryption_key/argon2_params, the encryption error variants, andadd_key/rotate_key/remove_key. Internal crypto/superblock items arepub(crate).encryption_keykwarg, the three rotation methods, and the six encryption exception classes.Some/Nonecipher branch).Testing
cargo clippy --all-targets -- -D warningsclean on both the lib andchisel-py.DecryptionFailed, anti-relocation, and a plaintext byte-identity regression guard.Threat model (documented boundaries)
Provides confidentiality, per-page and superblock tamper-detection, and anti-relocation. Does not provide rollback/replay resistance (an attacker substituting a wholly older, validly-signed image needs an external trust anchor to detect), in-memory DEK protection beyond zeroize-on-drop, or size/traffic-pattern hiding.
Deferred
Bulk DEK rotation — full re-encryption of every page under a fresh DEK, for the "the DEK itself is compromised" case — is deferred as a heavy whole-file operation (ISSUES.md I142). It is distinct from the implemented O(1) credential (KEK) rotation.