[enaboapps]

Closes #1073

Depends on switchifyapp/switchify-prediction#38 (library issue switchifyapp/switchify-prediction#37).

Do not merge yet.

• Temporary pins. switchify-prediction and switchify-prediction-neural are pinned to the library PR head 134eb800984a85c9aed86e160f57e670d39d6768. The worker assets are pinned to that PR's prepared CI artifacts: run 38756471848, BUILD.json commit 0148e0a3875296087dbbc3a4e57b5f9bcad52193. Re-pin the libraries and workers to the merged library commit or release tag before merging.
• Latency gate. The library PR records that a fresh long context breaches the 1,600 ms p95 bar on the portable worker. The window size is still to be decided.

What changes

• Context. prediction::context::extract gains text: all buffered text before the prefix, earlier sentences included. The statistical before is unchanged and still holds only the current sentence. In a clipped buffer, text starts at the first whole sentence, so its start moves only when a whole sentence is clipped; with no sentence end, only the clipped word is dropped, as before. partial handling is unchanged.
• Model. Predict::predict takes the Context. Model gives the statistical predictor before and gives generate/generate_phrases the text. The 16 KiB limits are the library's and are unchanged.
• Cache reuse. On a text change, a new request generation or an acceptance, the worker now calls the new Predict::cancel / Refiner::cancel instead of reset. Pending results are still dropped, but the companion keeps its prepared context, so the next keystroke extends it instead of evaluating up to 128 tokens again. Clearing the buffer (Enter, Tab, window change, outside input, keyboard close) still calls reset and discards the context. Buffer rules, learning and privacy are unchanged. Nothing is logged.
• Pins. Cargo.toml, Cargo.lock and scripts/prediction-neural.json are updated: bundle context_tokens 128, revision, library_revision, and prepared_artifact worker pins for all four targets. Model and tokenizer bytes are unchanged.
• Docs. docs/word-prediction.md and docs/neural-prediction.md are updated.

Tests

All tests use fakes; none type or move the pointer.

• context.rs: earlier sentences reach text, while before and prefix stay as today, including CJK sentence ends, clipping with and without a sentence end, and partial.
• model.rs: statistical words are identical whether or not earlier text is present, the phrase request keeps the full text, and cancel clears pending state.
• worker.rs: the predictor receives the current sentence and the full text while typing. Typing calls cancel and never reset; Enter resets.

Benchmarks

These are Windows figures from the library PR (generation_evaluate.py, 1,000 queries, AMD Ryzen AI 9 HX 370), all in ms.

Run	Median	p95	Max	Gen. failures	Six-slot top six
Portable, 512-char history, before	307	545	1,019	0	72.6%
Portable, 512-char history, 128 tokens	1,261	1,600	1,838	3	78.9%
Portable, 512-char history, 96 tokens	1,274	1,580	1,644	0	79.0%
AVX2, 512-char history, before	137	340	1,946	2	72.6%
AVX2, 512-char history, 128 tokens	502	881	2,027	41	76.6%
Portable, typing without resets, before	264	1,236	1,681	0
Portable, typing without resets, 128 tokens	247	490	1,982	10

• Fresh contexts. The history rows reset the context for every query, which is the worst case.
• Typing rows. These match the app after this change: the context is extended, apart from fresh prefills when the window moves.
• Noise. The machine was in desktop use, so failure counts include contention; even the unchanged AVX2 baseline timed out.
• Not run. I did not run docs/six-predictions.md's app integration benchmark, which resets each query with five short fixed texts and would not exercise long contexts. macOS was not measured.

Validation

All run locally on Windows with Node 24.19 and Rust 1.97.1:

• npm run prediction-model: the network fetch failed (ECONNRESET), so I seeded verified resources from the main checkout. After the re-pin, the script fetched the prepared workers and verified them offline.
• npm run lint passes.
• npm test passes: 300 vitest tests and 12 node tests.
• npm run build passes.
• cargo fmt --check passes.
• cargo clippy --all-targets -- -D warnings passes.
• cargo test passes: 796 passed, 5 ignored.

🤖 Generated with Claude Code

[enaboapps]

Closing without merging: #1073 was closed as won't fix on 2026-10-17.