Benchmark general-purpose neural word prediction

[enaboapps]

Result

The SmolLM2-135M reranker increased early top-five accuracy from 32.94% to 37.50% on the frozen general-purpose synthetic fixtures, a gain of 4.56 percentage points. Every early domain/prefix cell improved. It missed the speed target: CPU F32 p95 was 229.354 ms versus 6.029 ms for the current predictor. Peak process working set was about 1.01 GiB versus 371 MiB. Do not promote this implementation.

These are diagnostic fixtures, not representative user text or proof of unseen-user accuracy. AAC does not drive selection. The existing control retains its historical training mixture.

Change

Adds a separate unpublished Rust/Candle crate, pinned model downloads, a frozen 1,280-query-per-mode workload and aggregate results. It reranks the existing engine's top eight prefix-matching candidates using all subword tokens and an explicit word-boundary approximation. Messages, email, documents and search-style text are weighted equally. No personal learning or inference network access.

Also documents FUTO's official roughly 31 MB English model and the custom keyboard decoding path needed for a faithful standalone benchmark. FUTO was investigated, not benchmarked.

Production APIs, dependencies, model formats and Switchify PC are unchanged. No model weights or captured user text are committed or published.

Validation

• Production fmt, Clippy with warnings denied, 22 Rust tests and 12 Python tests passed locally.
• Separate experiment fmt, Clippy and three fake/pure Rust tests passed.
• Completed both 1,280-query runs with zero failures; source checksums verified.
• Production dependency audit passed. The isolated Candle lockfile audit passed with one documented unmaintained-package exception, RUSTSEC-2024-0436 for paste 1.0.15.
• Independent review of final head e1c8fb5 found no actionable findings.
• All latest-head CI passed: production checks and packaged smoke tests and experimental checks and audit. macOS runtime performance remains untested.

See docs/general-neural-results.md, docs/general-neural-results.json and docs/general-neural-protocol.md for measured tradeoffs and reproduction.

Closes #13.