feat: port rotortree

[rymnc]

brings rotortree into the workspace

[Copilot]

Pull request overview

This PR ports the rotortree crate into the workspace, adding an N-ary Lean Incremental Merkle Tree implementation plus optional persistence (WAL + checkpointing), along with extensive tests, benches, and examples.

Changes:

• Add crates/rotortree including core tree logic (LeanIMT), hashing adapters (BLAKE3 SIMD batching), and proof/persistence infrastructure.
• Add comprehensive test suites (unit + integration + proptest), benchmarks, and examples demonstrating usage.
• Update workspace metadata/docs and CI/Repo configuration to accommodate the new crate and workflows.

Reviewed changes

Copilot reviewed 40 out of 42 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
README.md	Adds workspace-level mention/link to the new rotortree crate.
Cargo.toml	Adds release/profiling profile settings for optimized builds.
.gitignore	Ignores additional Rust/benchmark artifacts introduced by the crate tooling/examples.
.github/workflows/ci.yml	Ensures PRs targeting main trigger CI.
.cargo/config.toml	Sets RUST_MIN_STACK for runtime needs (notably tests/benches).
crates/.gitkeep	Keeps the crates/ directory present in the repo layout.
crates/rotortree/Cargo.toml	Defines the new crate, features, deps (incl. pinned blake3), benches/examples.
crates/rotortree/README.md	Crate documentation covering design, usage, persistence, tuning, and dev workflows.
crates/rotortree/src/lib.rs	Crate root: feature gates, exports, lint settings, module wiring.
crates/rotortree/src/hash.rs	Defines Hash, HashState, and Hasher APIs including batched hashing hook.
crates/rotortree/src/error.rs	Defines TreeError and user-facing error messages for tree operations.
crates/rotortree/src/chunked_level.rs	Chunked storage layer enabling structural sharing, segments, and optional mmap backing.
crates/rotortree/src/tree.rs	Core LeanIMT logic, snapshotting, batch insert paths (incl. SIMD/parallel hooks).
crates/rotortree/src/test_util.rs	Test helper hasher + leaf generator utilities used across test suites.
crates/rotortree/src/adapters/mod.rs	Adapter module wiring (feature-gated BLAKE3 adapter).
crates/rotortree/src/adapters/blake3.rs	BLAKE3 hasher adapter with SIMD multi-input batching for eligible arities.
crates/rotortree/src/storage/mod.rs	Persistent RotorTree wrapper: WAL buffering, background flush/checkpoint threads, snapshots.
crates/rotortree/src/storage/config.rs	Defines RotorTreeConfig and FlushPolicy.
crates/rotortree/src/storage/error.rs	Storage/persistence error types (StorageError, RotorTreeError) and formatting.
crates/rotortree/src/storage/frame.rs	Length-prefixed, CRC32C-framed serialization primitives for WAL/checkpoint files.
crates/rotortree/src/storage/wal.rs	WAL header/entry schema and (de)serialization helpers.
crates/rotortree/src/storage/token.rs	Durability tracking token to wait for fsync of specific WAL seq numbers.
crates/rotortree/src/storage/recovery.rs	WAL (and checkpoint) recovery logic and tests.
crates/rotortree/src/storage/data.rs	Mmap region wrapper used for checkpoint-backed chunk remapping.
crates/rotortree/src/storage/checkpoint.rs	Checkpoint layout, metadata, tails, sharding, mmap loading, and atomic writes.
crates/rotortree/tests/lib.rs	Adds an integration-test crate root (currently empty).
crates/rotortree/tests/tree.rs	Integration tests covering correctness, boundaries, and feature-gated modes.
crates/rotortree/tests/proptest_tree.rs	Property-based tests for determinism, proofs, consistency proofs, and batching.
crates/rotortree/tests/simd_batch.rs	Proptest coverage ensuring SIMD-batched BLAKE3 hashing matches sequential roots.
crates/rotortree/tests/parallel.rs	Feature-gated integration tests for rayon-parallel batch insert behavior.
crates/rotortree/tests/concurrent.rs	Feature-gated integration tests for concurrent insert/snapshot/proof generation behavior.
crates/rotortree/proptest-regressions/simd_batch.txt	Stores known proptest regression seeds for reproducible failures.
crates/rotortree/examples/light_client.rs	End-to-end example demonstrating consistency proofs (full node vs light client).
crates/rotortree/examples/bulk_load.rs	Bulk-load benchmarking/example using persistent storage mode.
crates/rotortree/benches/common/mod.rs	Shared benchmark leaf generation helper.
crates/rotortree/benches/tree_bench.rs	Criterion benchmarks for in-memory tree operations across arities.
crates/rotortree/benches/tree_bench_parallel.rs	Criterion benchmarks for parallel insert_many (feature-gated).
crates/rotortree/benches/tree_bench_concurrent.rs	Criterion benchmarks for concurrent mode (feature-gated).
crates/rotortree/benches/tree_bench_all.rs	Criterion benchmarks covering combined feature scenarios (feature-gated).
crates/rotortree/benches/tree_bench_storage.rs	Criterion benchmarks for persistent RotorTree operations (feature-gated).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.