Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
17a73a3
docs: design spec for on-disk encryption (XChaCha20-Poly1305, envelop…
Xof Jun 30, 2026
8564e38
docs: implementation plan for on-disk encryption (6 phases, 29 tasks)
Xof Jun 30, 2026
6b3c33e
build: add RustCrypto deps and register crypto core module
Xof Jun 30, 2026
845948e
feat(error): add encryption error variants (NoEncryptionKey, InvalidE…
Xof Jun 30, 2026
8269f22
feat(crypto): key types, KdfId, Argon2Params, CryptoError, OS randomness
Xof Jun 30, 2026
758dca2
feat(crypto): derive_kek dispatching HKDF-SHA256 and Argon2id
Xof Jun 30, 2026
70f24c5
fix(crypto): prevent key-byte leakage in Kek Debug and KDF output buffer
Xof Jun 30, 2026
aa90bc0
feat(crypto): DEK wrap/unwrap under KEK with detached XChaCha20-Poly1305
Xof Jun 30, 2026
a02b2ec
fix(crypto): zeroize decrypted DEK plaintext from open_detached
Xof Jun 30, 2026
196c04e
feat(crypto): PageCipher whole-page and body seal/open
Xof Jun 30, 2026
9b1fa71
test(crypto): zeroization guards; clippy/fmt clean for crypto core
Xof Jun 30, 2026
8e9ca00
feat(superblock): add crypto-header key-slot table in the reserved re…
Xof Jun 30, 2026
44e9aa2
feat(superblock): DEK-sealed body and Superblock.encryption field
Xof Jun 30, 2026
700f0d0
fix(superblock): bounds-check sealed-body ct_len before slicing
Xof Jun 30, 2026
ccd523b
feat(engine): create encrypted database — wrap DEK into slot 0, stamp…
Xof Jun 30, 2026
9a2a663
test(superblock): assert encrypted named_roots name is absent from cl…
Xof Jun 30, 2026
b0cfd06
feat(engine): open encrypted database — unwrap DEK from key-slot, dec…
Xof Jun 30, 2026
7810428
fix(recovery): select winning superblock buffer directly for encrypte…
Xof Jun 30, 2026
be3993e
test(engine): verify session DEK lifetime — cipher Some/None structur…
Xof Jun 30, 2026
815a9e1
feat(page_io): stride-aware raw on-disk page units (8232 encrypted, 8…
Xof Jun 30, 2026
c3faf2b
feat(spillway): parameterize slot by payload_size for sealed-blob sup…
Xof Jun 30, 2026
37ba983
feat(page_cache): wire PageCipher seal/open + uniform 8232 stride (Ta…
Xof Jun 30, 2026
c66479b
feat(page_cache): seal-once on evict-to-spillway; verbatim copy on drain
Xof Jun 30, 2026
63b03e5
feat(api): expose Key/Argon2Params publicly, add argon2_params to Opt…
Xof Jun 30, 2026
866078b
feat(api): wire encryption_key through open_in_memory + public-API ro…
Xof Jul 1, 2026
c9206a4
feat(python): add encryption_key kwarg to open() + exception classes
Xof Jul 1, 2026
7b754db
test(python): pin encryption exception contract; trim zeroize feature
Xof Jul 1, 2026
2903316
feat(crypto_header): slot-table helpers — unlock/free_slot/wrap_into
Xof Jul 1, 2026
d218a36
feat(crypto): rewrite_crypto_header metadata-only superblock commit
Xof Jul 1, 2026
6ba7277
feat(crypto): Chisel::add_key and rotate_key
Xof Jul 1, 2026
0d3fce9
feat(crypto): Chisel::remove_key with last-slot guard
Xof Jul 1, 2026
1c0d981
feat(python): bind add_key/rotate_key/remove_key; add NoFreeKeySlotEr…
Xof Jul 1, 2026
421c1ac
feat(format): add encrypted-DB MAJOR version constant and gate-reject…
Xof Jul 1, 2026
8c0890a
refactor(format): make format_version_encrypted the single source for…
Xof Jul 1, 2026
15d2b3e
docs: document on-disk encryption architecture and defer bulk DEK rot…
Xof Jul 1, 2026
00a5b93
refactor(api): scope internal crypto/format/superblock items to pub(c…
Xof Jul 1, 2026
bbdabfa
fix(crypto): return typed errors instead of panicking on bad input/co…
Xof Jul 1, 2026
73f5b3d
docs: fix encryption comments and ARCHITECTURE prefix layout
Xof Jul 1, 2026
e0c72d3
refactor(test): relocate internal-crypto tests inline after pub(crate…
Xof Jul 1, 2026
98c1815
test(crypto): Argon2id KAT, in-memory encryption, rewrite_crypto_head…
Xof Jul 1, 2026
9fff049
Merge branch 'main' into feature/on-disk-encryption
Xof Jul 1, 2026
6c93636
style: rustfmt the encryption-branch files
Xof Jul 1, 2026
e59fdd7
build: hold the 1.82 MSRV floor for the new crypto deps
Xof Jul 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ This is a living document; update it when the architecture changes. Decisions do
5. [Recovery on open](#recovery-on-open)
6. [On-disk format](#on-disk-format)
7. [Cross-cutting concepts](#cross-cutting-concepts)
- [On-disk encryption](#on-disk-encryption)
8. [Benchmark infrastructure](#benchmark-infrastructure)
9. [Implementation history](#implementation-history)
10. [Glossary](#glossary)
Expand Down Expand Up @@ -615,6 +616,37 @@ Chisel versions its on-disk format at two levels.

Both schemes leave reserved space for forward compatibility — the superblock has bytes 324..8184 reserved (after the `freemap_depth` field at 320..324), and every non-superblock page has bytes 8..16 reserved (8 bytes / 64 bits) for future common-header fields.

### On-disk encryption

Chisel supports optional authenticated encryption of database files. An encrypted database is indistinguishable from random bytes to a reader without the key; each page is individually authenticated, so corruption (accidental or deliberate) is detected before any plaintext is returned.

**Cipher.** XChaCha20-Poly1305 (IETF extended-nonce variant). Each page write generates a fresh random 192-bit nonce; the extended nonce space (2¹⁹²) makes nonce reuse under shadow-paging page reassignment negligible in practice (spec §2.1). The on-disk layout per encrypted page is `ciphertext(8192) ‖ tag(16) ‖ nonce(24)` = 8232 bytes (`ENC_PAGE_SIZE`). The additional data (AAD) for each page is the 8-byte little-endian `page_id`, which binds ciphertext to its slot and prevents a valid block from being relocated to another page position without detection.

**On-disk stride.** Encrypted databases use a uniform 8232-byte stride for every page including the superblock slots. Plaintext databases continue to use the 8192-byte stride; the two are mutually exclusive and the stride is recorded in the superblock's plaintext crypto-header so the engine reads the correct number of bytes before attempting any operation. The `page_io` layer is stride-agnostic: callers set the stride once (via `PageIo::set_stride`) and all subsequent raw reads and writes use it.

**Envelope (key hierarchy).** A random 256-bit per-database encryption key (DEK) encrypts all page content. The DEK itself is never stored in plaintext: it is wrapped under a key-encryption key (KEK) and the wrapped form is held in a plaintext key-slot table inside the superblock's reserved region (bytes 324..1356; 8 slots × 128 bytes each, preceded by an 8-byte prefix (1-byte algorithm id, 4-byte stride, 3 reserved bytes); the key-slot table begins at byte 332). Each slot stores the KDF identity, KDF parameters, salt, wrap nonce, wrapped DEK, and wrap tag. There are two KEK derivation paths:

- **Raw key** (`Key::Raw`): KEK = HKDF-SHA256(ikm=key material, salt=slot salt, info=`"chisel-kek"`).
- **Passphrase** (`Key::Passphrase`): KEK = Argon2id(password, salt, m/t/p from the slot's stored parameters).

The DEK wrapping uses detached XChaCha20-Poly1305 with AAD bound to the slot's KDF metadata, so an attacker cannot swap a slot's KDF parameters to force mis-derivation without breaking the tag.

**Superblock body protection.** The superblock's sensitive body — root pointers (`root_handle_table`, `root_freemap_page`, `root_tag_map_page`), `total_pages`, `next_handle`, `freemap_depth`, and the `named_roots` name table — is sealed under the DEK as a `nonce ‖ tag ‖ ciphertext` sub-blob whose AAD binds it to the superblock's identity. The plaintext portion of the superblock (magic, format version, txn counter, page size, superblock count, crypto-header) retains its XXH3 checksum so the A/B torn-write selector (`select()`) still works before any decryption.

**Format version.** Encrypted databases stamp file-level **MAJOR = 2, MINOR = 0**. Plaintext databases remain at MAJOR = 1. The existing open-time gate (which rejects any file whose MAJOR differs from the compiled-in `FORMAT_MAJOR_VERSION`) therefore hard-rejects an encrypted database on an encryption-unaware binary with `UnsupportedFormatVersion`, preventing ciphertext from being silently misread as page data. No per-page (I31) format change is needed — the logical page image is unchanged.

**Key management.** Credential rotation is O(1) and crash-safe — it never re-encrypts any page:

- `add_key(old_key, new_key)`: derives a new KEK, wraps the same DEK into a free slot, then commits.
- `rotate_key(old_key, new_key)`: `add_key` followed by clearing the old slot in the same commit.
- `remove_key(key)`: clears the matching slot, refusing to clear the last active slot (which would make the database permanently unreadable).

Each operation is a normal superblock commit through the A/B + fsync protocol. Bulk DEK rotation (re-encrypting every page under a fresh DEK — relevant only when the DEK itself is believed compromised) is deferred; see I142.

**Spillway.** For encrypted databases the in-memory spillway carries sealed blobs: pages are encrypted exactly once on eviction from the page cache (`seal` on evict-to-spillway) and copied verbatim — without decryption or re-encryption — on drain to the main file. Rehydration from the spillway decrypts the blob back into the cache. This means no plaintext page content is ever written to disk by an encrypted database, even during spill.

**Threat-model boundary.** Provided: confidentiality of all user data and sensitive metadata at rest; AEAD tamper-detection per page and per superblock body (any modification surfaces as the fatal `DecryptionFailed` error, which poisons the engine); anti-relocation (AAD = `page_id` prevents transplanting a ciphertext block to a different slot). Not provided: rollback/replay resistance (an attacker who substitutes a wholly older, validly-signed database image cannot be detected without an external monotonic trust anchor such as a TPM); in-memory protection beyond `zeroize`-on-drop for the DEK and page plaintext; traffic-analysis resistance (file size, page count, and access patterns are visible).

---

## Benchmark infrastructure
Expand Down
Loading
Loading