diff --git a/docs/neural-prediction.md b/docs/neural-prediction.md index e4e0828a..e501db66 100644 --- a/docs/neural-prediction.md +++ b/docs/neural-prediction.md @@ -37,3 +37,7 @@ The benchmark now requires 1,000 successful warmed refinements across at most fi ## Reply timeout update The companion inference and reset reply deadline is now 2 seconds, increased from 500 ms after repeated macOS CI timeouts. Startup remains bounded at 30 seconds, statistical suggestions remain immediate, and late or stale results remain rejected. That change pinned both Rust libraries to `171210ee89e3944d1c606d50ad779a5dfb1651b8`, the timeout fix in switchify-prediction PR #22, and kept the v0.2.0 worker assets. The v0.2.1 worker assets from `2a2ed0c2b0013663ffa9317dba3389ec96b6f14b` added six-slot generation. The worker assets, their converter and both Rust libraries now come from the v0.2.2 release, `b873c88314b0eb0573f3c34d301fbd2f5761d2ab`, still on worker protocol 2. Its workers batch the search and stop it as soon as the ranked words are settled. For the same context they return the same words 3–5× faster: a median of 581 to 121 ms with AVX2 and 1,257 to 424 ms portable on a Windows test machine, and 690 to 151 ms on macOS ARM CI in single runs, whose timing varies. Generation now sees the current sentence as typed, with case and punctuation. The libraries add incremental, bounded personal learning. The deadlines are unchanged. The worker assets, converter and both Rust libraries now come from the v0.3.0 release, `1b5fc54610545124dcca206242a512909add139e`, on worker protocol 3. It adds two-word phrase generation, which fills slots 6–8. Word ranking and generation are unchanged, and so are the model, tokenizer and deadlines. The earlier 500 ms measurements above remain historical evidence, not measurements of the increased deadline. + +## Earlier sentences + +Generation now sees the earlier sentences in the buffer, up to a 128-token window, and the worker keeps its prepared context between keystrokes instead of evaluating it again. Both Rust libraries are pinned to `134eb800984a85c9aed86e160f57e670d39d6768`, the head of switchify-prediction PR #38, and the worker assets come from that PR's prepared CI artifacts (run 38756471848, built from merge commit `0148e0a3875296087dbbc3a4e57b5f9bcad52193`). The bundle manifest records `context_tokens` 128; model and tokenizer bytes are unchanged. These pins are temporary and must move to the merged library release before this ships. diff --git a/docs/word-prediction.md b/docs/word-prediction.md index bae6f539..b8c8f086 100644 --- a/docs/word-prediction.md +++ b/docs/word-prediction.md @@ -25,7 +25,7 @@ A passive observer records only an activity counter and timestamp, never externa One Word prediction setting controls the English `en-aac-oanc-v1` SQLite database and the SmolLM2 model through the pinned `switchify-prediction` Rust libraries. The saved `enhancedWordPrediction` field remains compatible but does not choose an engine. Only the isolated worker opens the read-only baseline and the personal file described in [Learning from typing](#learning-from-typing); there is no import or runtime download. -The worker ranks up to three words using interpolated trigram/bigram/unigram probabilities (0.6/0.3/0.1, renormalized when context is unavailable) and shows them immediately in slots 1–3. The SmolLM2 model then fills slots 4 and 5 with up to two different words, and afterwards slots 6–8 with its three best two-word phrases; see [default neural-assisted prediction](neural-prediction.md). Suggestions begin with the first typed character and continue after a space as next-word suggestions. An untouched keyboard remains empty. The prefix and preceding text are supplied separately, and sentence punctuation resets language context. Phrases are the only suggestions of more than one word. A phrase's first word completes the word in progress, or is the next word, and it never crosses sentence punctuation. A prefix the database does not know leaves slots 1–3 empty; the model may still offer words and phrases for slots 4–8. A suggestion whose accepted text would exceed the 64 characters the executor types is not offered. The vocabulary can contain names, disfluencies and inappropriate words; it is English-only and is not a clinical recommendation system. +The worker ranks up to three words using interpolated trigram/bigram/unigram probabilities (0.6/0.3/0.1, renormalized when context is unavailable) and shows them immediately in slots 1–3. The SmolLM2 model then fills slots 4 and 5 with up to two different words, and afterwards slots 6–8 with its three best two-word phrases; see [default neural-assisted prediction](neural-prediction.md). Suggestions begin with the first typed character and continue after a space as next-word suggestions. An untouched keyboard remains empty. The prefix and preceding text are supplied separately. For slots 1–3, sentence punctuation resets language context. For slots 4–8, the model also sees the earlier sentences in the buffer, as typed, up to its 128-token window; a longer text is cut at the start of a sentence where one fits. When the buffer has been clipped, the model's text starts at its first whole sentence, or after the clipped word when no sentence ends in it. Between keystrokes the model keeps the context it has evaluated and evaluates only what changed; the context is discarded whenever the buffer is cleared. Phrases are the only suggestions of more than one word. A phrase's first word completes the word in progress, or is the next word, and it never crosses sentence punctuation. A prefix the database does not know leaves slots 1–3 empty; the model may still offer words and phrases for slots 4–8. A suggestion whose accepted text would exceed the 64 characters the executor types is not offered. The vocabulary can contain names, disfluencies and inappropriate words; it is English-only and is not a clinical recommendation system. Loading is asynchronous and typing remains available. The existing badge distinguishes loading, waiting for typed context, held suggestions, no match, available suggestions, paused activity tracking, and failure. **Retry predictions** replaces the worker without closing the keyboard. A clipped buffer without a complete earlier word yields no suggestions. Repeated calls exceeding 1.5 seconds mark prediction unavailable; a missed two-second reply deadline triggers the existing worker replacement policy. Errors never expose buffered text. diff --git a/scripts/prediction-neural.json b/scripts/prediction-neural.json index a94d6264..3b6c6016 100644 --- a/scripts/prediction-neural.json +++ b/scripts/prediction-neural.json @@ -1,5 +1,5 @@ { - "revision": "1b5fc54610545124dcca206242a512909add139e", + "revision": "0148e0a3875296087dbbc3a4e57b5f9bcad52193", "bundle": { "format_version": 1, "model_id": "smollm2-135m-q8-v1", @@ -7,7 +7,7 @@ "revision": "93efa2f097d58c2a74874c7e644dbc9b0cee75a2", "conversion": "Candle 0.11.0 Q8_0, F32 norms, adjacent-pair RoPE, tied embeddings; neural/worker/src/bin/quantize.rs", "policy": { - "context_tokens": 64, + "context_tokens": 128, "shortlist": 8, "max_results": 5, "scoring": "whole-word log probability plus boundary probability, sequential", @@ -84,11 +84,16 @@ }, "workers": { "aarch64-apple-darwin": { - "archive_sha256": "8b936166e30be1ed4fe1385b062cbb5b9569bbcdfe11d414752c52fe1f240eae", + "prepared_artifact": { + "run_id": 38756471848, + "name": "neural-macos-latest", + "archive": "switchify-prediction-neural-0.3.0-aarch64-apple-darwin.zip" + }, + "archive_sha256": "9c0c6a1198279b033091db2b20e228a05e5a44d45a5b9fc0a4fe334dcc3a0b4e", "files": { "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/BUILD.json": { "size": 987, - "sha256": "99209db6797bb5ff5662b2a66bea542595f0377a8d885acd062452847f7fe5bb" + "sha256": "41164ec057107cd993e2150f13d4d236de654f64331dec813f5e122ae125bc49" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/Cargo.lock": { "size": 46276, @@ -107,8 +112,8 @@ "sha256": "f5d13a7406728515112cab4da093ee19642f68abbb9f30c8c8f511ae68ec8eb3" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/README.md": { - "size": 9936, - "sha256": "939722a156350cb1618a43f05290ab2c4c444f65448c9d47c26e107866b9cac4" + "size": 10853, + "sha256": "ec01c9f79f442811985ceb7be73f92ec2b963cde6fae884904cc5dcb6c44440b" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/RUST_LIBRARY_COPYRIGHT.html": { "size": 279302, @@ -120,29 +125,33 @@ }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/SHA256SUMS": { "size": 998, - "sha256": "6b228174a3f63ceb469ba6807d7f5228b4761ee35f05cd6fc191529980641891" + "sha256": "90e3089515106eb27b4e7d82bdc46e88187dd8ba90282000bc95304bfcecb55e" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/THIRD_PARTY_NOTICES.md": { "size": 1545871, "sha256": "6bc01a7be7b3aaa59d2c04acbbdea91b20699f0f20d556debc5e674eb0d3600f" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/switchify-smol-worker": { - "size": 7172016, - "sha256": "d557d9748b977a6af5bdace5b2b54ff949e984c7ca3a30652d8fef73556f14b7" + "size": 7184848, + "sha256": "7858c78a3b8e0f52bddf13ab9f0f0741917d5982b8574e2032c7ad5f0f8dcade" }, "switchify-prediction-neural-0.3.0-aarch64-apple-darwin/verify_bundle.py": { "size": 1282, "sha256": "69c124d037d9e25018f690c10b861c4388c01a8cd8eb3ed438d17136196c1194" } - }, - "url": "https://github.com/switchifyapp/switchify-prediction/releases/download/v0.3.0/switchify-prediction-neural-0.3.0-aarch64-apple-darwin.zip" + } }, "x86_64-apple-darwin": { - "archive_sha256": "970082e5d1a0622c7c81d155b538896c61e858e00d2dc59bf443517264c5d3dc", + "prepared_artifact": { + "run_id": 38756471848, + "name": "neural-macos-15-intel", + "archive": "switchify-prediction-neural-0.3.0-x86_64-apple-darwin.zip" + }, + "archive_sha256": "3d79246e4a736c9732f1a3dae85625f17f1780a2dc7c5ae3d50c58f7703e996c", "files": { "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/BUILD.json": { "size": 1024, - "sha256": "5a6d6869c2d4b1ac63fd0069688aa50a36377694607f59d1ce7da0971c468ec0" + "sha256": "e2ad4643ba1873262cd86b68ab1f1c681e300cfa3de2d5098eab588fdd119a60" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/Cargo.lock": { "size": 46276, @@ -161,8 +170,8 @@ "sha256": "f5d13a7406728515112cab4da093ee19642f68abbb9f30c8c8f511ae68ec8eb3" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/README.md": { - "size": 9936, - "sha256": "939722a156350cb1618a43f05290ab2c4c444f65448c9d47c26e107866b9cac4" + "size": 10853, + "sha256": "ec01c9f79f442811985ceb7be73f92ec2b963cde6fae884904cc5dcb6c44440b" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/RUST_LIBRARY_COPYRIGHT.html": { "size": 279302, @@ -174,33 +183,37 @@ }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/SHA256SUMS": { "size": 1091, - "sha256": "0c55204ca19b7d2caaecdb7a8fb74ca7a4c32d0e2674c6441f42bce51addfcbe" + "sha256": "1c8801d1556f98a64be52a5be8466ea5e8afffdba46cf9b4be94b3bacc278b16" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/THIRD_PARTY_NOTICES.md": { "size": 1545871, "sha256": "6bc01a7be7b3aaa59d2c04acbbdea91b20699f0f20d556debc5e674eb0d3600f" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/switchify-smol-worker": { - "size": 7718240, - "sha256": "04fa318763274ba502bfc2c48d9c120c578d467fdccc4659400ad3411f42a79d" + "size": 7727216, + "sha256": "8fe5b82397497859712eec481e68922f2633049e0229e582be1ffabdb2f8ad4e" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/switchify-smol-worker-avx2": { - "size": 7758544, - "sha256": "280bf4692bc25bf4e0950b94ff1cc6d7b260bfaa0614b128827bfbc6beb5a07c" + "size": 7763320, + "sha256": "fafc5ab59ff661b8f1366a9b629e268fda8413360a8485e2a78fd3d1a9b24ea3" }, "switchify-prediction-neural-0.3.0-x86_64-apple-darwin/verify_bundle.py": { "size": 1282, "sha256": "69c124d037d9e25018f690c10b861c4388c01a8cd8eb3ed438d17136196c1194" } - }, - "url": "https://github.com/switchifyapp/switchify-prediction/releases/download/v0.3.0/switchify-prediction-neural-0.3.0-x86_64-apple-darwin.zip" + } }, "x86_64-pc-windows-msvc": { - "archive_sha256": "0d9606705bc8555eae09997d4ce4ab571ec8a6f5dfff7629860ace4d4241c41f", + "prepared_artifact": { + "run_id": 38756471848, + "name": "neural-windows-latest", + "archive": "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc.zip" + }, + "archive_sha256": "ca43e76a6a805d8122d9468026a3f147d0ddc822a5b35eecbe524404ca928611", "files": { "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/BUILD.json": { "size": 1059, - "sha256": "94a04861c8645f3faa11ecbffa2b68668a39a5dc91d7cbe59d31be90ae1f4e7e" + "sha256": "8c885fd94b358d009fcc27ac89ee3d140b490f202ce08fb8170878108975c63e" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/Cargo.lock": { "size": 46276, @@ -219,8 +232,8 @@ "sha256": "934945362415a5672a5c05887553911778f2032508b5efcb9a755c5e551548dc" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/README.md": { - "size": 9936, - "sha256": "939722a156350cb1618a43f05290ab2c4c444f65448c9d47c26e107866b9cac4" + "size": 10853, + "sha256": "ec01c9f79f442811985ceb7be73f92ec2b963cde6fae884904cc5dcb6c44440b" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/RUST_LIBRARY_COPYRIGHT.html": { "size": 279302, @@ -232,33 +245,37 @@ }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/SHA256SUMS": { "size": 1116, - "sha256": "a265ec20990c0416248b0299e629d3bc61977a538305a7a0caa5655bcf59914d" + "sha256": "025309a13fc25af9e1e374cc6c50ffe9b3a89a205a9fe59f3f1ff4db3dc44c64" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/THIRD_PARTY_NOTICES.md": { "size": 1552635, "sha256": "448c7f55a9ef918d5cf67f57cfb141c62f7bc6d819cc168fc4ad20f73b34c5f0" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/switchify-smol-worker-avx2.exe": { - "size": 8379392, - "sha256": "40ce12dd15b99cef50900bcfee2c4440cbc3c42d69aa298a802d9820ff6e39a7" + "size": 8381952, + "sha256": "e8da56fe16fe2a9b3e7f03646aa9dd9e696afd761d8200f899faf78fbfe65301" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/switchify-smol-worker.exe": { - "size": 8296448, - "sha256": "e4d5d84297af39b3c74397b7639ea2ecbf96dfd2f5285c02c56e9adeac68ecea" + "size": 8297472, + "sha256": "8bcbc2d8c33d854c945362a7e7adf173a8c3b31ef5808253a048c04da1f3c4c2" }, "switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc/verify_bundle.py": { "size": 1315, "sha256": "96b47d695b104c61ca65de856108d8850a6da7c00220ae72dd91b8e73cb8f6f2" } - }, - "url": "https://github.com/switchifyapp/switchify-prediction/releases/download/v0.3.0/switchify-prediction-neural-0.3.0-x86_64-pc-windows-msvc.zip" + } }, "x86_64-unknown-linux-gnu": { - "archive_sha256": "d6da6b590690cae168d5c3b47273ba6c1d2ce361bcc291cc3a9a3d8a52176f43", + "prepared_artifact": { + "run_id": 38756471848, + "name": "neural-ubuntu-latest", + "archive": "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu.zip" + }, + "archive_sha256": "ef9d1fb3e13edcbc561eb357324a29c7b315710c622f35c41e3251764084dfa2", "files": { "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/BUILD.json": { "size": 1051, - "sha256": "babe4c2e63e246d64e5f64a395286e143a75e2bb9bf2ed136771f2ac3dea3736" + "sha256": "96d266be3fb11355eeb1c9c69c49f6889af06cade8ebd602546ea353096a0a0d" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/Cargo.lock": { "size": 46276, @@ -277,8 +294,8 @@ "sha256": "f5d13a7406728515112cab4da093ee19642f68abbb9f30c8c8f511ae68ec8eb3" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/README.md": { - "size": 9936, - "sha256": "939722a156350cb1618a43f05290ab2c4c444f65448c9d47c26e107866b9cac4" + "size": 10853, + "sha256": "ec01c9f79f442811985ceb7be73f92ec2b963cde6fae884904cc5dcb6c44440b" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/RUST_LIBRARY_COPYRIGHT.html": { "size": 279302, @@ -290,27 +307,26 @@ }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/SHA256SUMS": { "size": 1091, - "sha256": "f3332ca23410d7857b8f7df5c5baea23289d401dfd03d4bdfb95a8688cbaac2a" + "sha256": "f1d88a324838fd13b2b2e7f1334049166fee60e29cf414f29f1c869e1f28026f" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/THIRD_PARTY_NOTICES.md": { "size": 1535176, "sha256": "93cad9c9282edec579015f6d822901ab48596ff21e9f74bb3f07dc2ae8b7314c" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/switchify-smol-worker": { - "size": 9978872, - "sha256": "08253b9d56aa9ea10f18fcc430c3b6f88e61d6dc093d7ccfa4341d8b5000fc15" + "size": 9985984, + "sha256": "db73c62deb2f3cdf67644242f5e34e32563e12220e6dbb9747df1ad9269ba0b4" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/switchify-smol-worker-avx2": { - "size": 9953304, - "sha256": "fce9db2cc8328c18068067330c63dad608a035675742fae406c626557a7d4d41" + "size": 9959448, + "sha256": "08e2335544b135e3ce4d918f71be8f0caaa8dee41b5535e7c6baf6598bc07cf4" }, "switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu/verify_bundle.py": { "size": 1282, "sha256": "69c124d037d9e25018f690c10b861c4388c01a8cd8eb3ed438d17136196c1194" } - }, - "url": "https://github.com/switchifyapp/switchify-prediction/releases/download/v0.3.0/switchify-prediction-neural-0.3.0-x86_64-unknown-linux-gnu.zip" + } } }, - "library_revision": "1b5fc54610545124dcca206242a512909add139e" + "library_revision": "134eb800984a85c9aed86e160f57e670d39d6768" } diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock index 40d20a05..e10d33c7 100644 --- a/src-tauri/Cargo.lock +++ b/src-tauri/Cargo.lock @@ -1032,7 +1032,7 @@ dependencies = [ "libc", "option-ext", "redox_users 0.5.2", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -1249,7 +1249,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -2547,7 +2547,7 @@ dependencies = [ "png 0.18.1", "serde", "thiserror 2.0.19", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -3318,7 +3318,7 @@ dependencies = [ "once_cell", "socket2", "tracing", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -3588,7 +3588,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -3646,7 +3646,7 @@ dependencies = [ "security-framework", "security-framework-sys", "webpki-root-certs", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -4051,7 +4051,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" dependencies = [ "libc", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -4217,7 +4217,7 @@ dependencies = [ [[package]] name = "switchify-prediction" version = "0.3.0" -source = "git+https://github.com/switchifyapp/switchify-prediction?rev=1b5fc54610545124dcca206242a512909add139e#1b5fc54610545124dcca206242a512909add139e" +source = "git+https://github.com/switchifyapp/switchify-prediction?rev=134eb800984a85c9aed86e160f57e670d39d6768#134eb800984a85c9aed86e160f57e670d39d6768" dependencies = [ "clap", "rusqlite", @@ -4233,7 +4233,7 @@ dependencies = [ [[package]] name = "switchify-prediction-neural" version = "0.3.0" -source = "git+https://github.com/switchifyapp/switchify-prediction?rev=1b5fc54610545124dcca206242a512909add139e#1b5fc54610545124dcca206242a512909add139e" +source = "git+https://github.com/switchifyapp/switchify-prediction?rev=134eb800984a85c9aed86e160f57e670d39d6768#134eb800984a85c9aed86e160f57e670d39d6768" dependencies = [ "clap", "serde", @@ -4679,7 +4679,7 @@ dependencies = [ "getrandom 0.3.4", "once_cell", "rustix", - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] @@ -5076,7 +5076,7 @@ dependencies = [ "png 0.18.1", "serde", "thiserror 2.0.19", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -5105,7 +5105,7 @@ checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e" dependencies = [ "memoffset", "tempfile", - "windows-sys 0.61.2", + "windows-sys 0.60.2", ] [[package]] @@ -5508,7 +5508,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.61.2", + "windows-sys 0.59.0", ] [[package]] diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml index cd359ad2..2ac1c88c 100644 --- a/src-tauri/Cargo.toml +++ b/src-tauri/Cargo.toml @@ -14,8 +14,8 @@ crate-type = ["staticlib", "cdylib", "rlib"] tauri-build = { version = "2", features = [] } [dependencies] -switchify-prediction = { git = "https://github.com/switchifyapp/switchify-prediction", rev = "1b5fc54610545124dcca206242a512909add139e" } -switchify-prediction-neural = { git = "https://github.com/switchifyapp/switchify-prediction", rev = "1b5fc54610545124dcca206242a512909add139e" } +switchify-prediction = { git = "https://github.com/switchifyapp/switchify-prediction", rev = "134eb800984a85c9aed86e160f57e670d39d6768" } +switchify-prediction-neural = { git = "https://github.com/switchifyapp/switchify-prediction", rev = "134eb800984a85c9aed86e160f57e670d39d6768" } anyhow = "1" base64 = "0.22" directories = "6.0.0" diff --git a/src-tauri/src/prediction/context.rs b/src-tauri/src/prediction/context.rs index c5e8d592..c1e20372 100644 --- a/src-tauri/src/prediction/context.rs +++ b/src-tauri/src/prediction/context.rs @@ -1,31 +1,49 @@ use unicode_segmentation::UnicodeSegmentation; +const SENTENCE_ENDS: [char; 8] = ['.', '!', '?', '\n', '\r', '。', '!', '?']; + pub struct Context { pub prefix: String, - /// Buffered text before `prefix`, without a clipped leading fragment. + /// The current sentence before `prefix`, without a clipped leading + /// fragment. Statistical prediction sees only this. pub before: String, + /// All buffered text before `prefix`, earlier sentences included, for + /// generation. When the buffer was clipped it starts at its first whole + /// sentence, or after the clipped word if no sentence ends in it, so its + /// start moves only when a whole sentence is clipped. + pub text: String, /// Clipping left no complete earlier word, so an empty `before` is not /// the true start of the text. pub partial: bool, } pub fn extract(text: &str, clipped: bool) -> Context { let context = extract_words(text, clipped); - let boundary = text.rfind(['.', '!', '?', '\n', '\r', '。', '!', '?']); + let boundary = text.rfind(SENTENCE_ENDS); let start = boundary.map_or(0, |i| i + text[i..].chars().next().unwrap().len_utf8()); - let mut before = &text[start..text.len() - context.prefix.len()]; + let typed = &text[..text.len() - context.prefix.len()]; + let mut before = &typed[start..]; if clipped && start == 0 { before = before .find(char::is_whitespace) .map_or("", |i| &before[i..]); } + let earlier = if clipped { + match typed.find(SENTENCE_ENDS) { + Some(i) => &typed[i + typed[i..].chars().next().unwrap().len_utf8()..], + None => before, + } + } else { + typed + }; Context { partial: clipped && start == 0 && before.trim().is_empty(), before: before.to_owned(), + text: earlier.to_owned(), ..context } } fn extract_words(text: &str, clipped: bool) -> Context { - let boundary = text.rfind(['.', '!', '?', '\n', '\r', '。', '!', '?']); + let boundary = text.rfind(SENTENCE_ENDS); let start = boundary.map_or(0, |i| i + text[i..].chars().next().unwrap().len_utf8()); let tail = &text[start..]; let mut tokens: Vec<_> = tail.unicode_word_indices().collect(); @@ -45,6 +63,7 @@ fn extract_words(text: &str, clipped: bool) -> Context { Context { prefix, before: String::new(), + text: String::new(), partial: false, } } @@ -70,4 +89,31 @@ mod tests { assert!(!extract("agment. h", true).partial); assert_eq!(extract("I can't", false).prefix, "can't"); } + + #[test] + fn generation_sees_earlier_sentences_and_statistics_do_not() { + let c = extract("We went to the beach. It was warm! The wa", false); + assert_eq!(c.prefix, "wa"); + assert_eq!(c.before, " The "); + assert_eq!(c.text, "We went to the beach. It was warm! The "); + let c = extract("Hi.\nOld context。I need h", false); + assert_eq!(c.before, "I need "); + assert_eq!(c.text, "Hi.\nOld context。I need "); + // Just after a sentence end there is no current sentence yet. + let c = extract("It was warm. ", false); + assert_eq!((c.prefix.as_str(), c.before.as_str()), ("", " ")); + assert_eq!(c.text, "It was warm. "); + // A clipped buffer starts at its first whole sentence. + let c = extract("ach. It was warm. The wa", true); + assert_eq!(c.before, " The "); + assert_eq!(c.text, " It was warm. The "); + assert!(!c.partial); + let c = extract("ach. The wa", true); + assert_eq!((c.before.as_str(), c.text.as_str()), (" The ", " The ")); + // Without a sentence end, only the clipped word is dropped. + let c = extract("agment whole pa", true); + assert_eq!((c.before.as_str(), c.text.as_str()), (" whole ", " whole ")); + let c = extract("agment", true); + assert!(c.partial && c.text.is_empty()); + } } diff --git a/src-tauri/src/prediction/database.rs b/src-tauri/src/prediction/database.rs index 5aeafac3..02f05fe2 100644 --- a/src-tauri/src/prediction/database.rs +++ b/src-tauri/src/prediction/database.rs @@ -79,6 +79,14 @@ impl Database { pub fn pending(&self) -> bool { matches!(&self.state, State::Ready(model) if model.pending()) } + /// Drop pending results while the same text continues. The generation + /// context stays, so the next request only evaluates what changed. + pub fn cancel(&mut self) { + if let State::Ready(model) = &mut self.state { + model.cancel(); + } + } + /// Drop pending results and the generation context with the text. pub fn reset(&mut self) { if let State::Ready(model) = &mut self.state { model.reset(); @@ -99,7 +107,7 @@ impl Database { unreachable!() }; let start = Instant::now(); - let result = model.predict(&context.before, &context.prefix); + let result = model.predict(context); self.slow_calls = if start.elapsed() > self.slow_call { self.slow_calls + 1 } else { @@ -146,7 +154,8 @@ impl Database { struct FakeModel; #[cfg(test)] impl Predict for FakeModel { - fn predict(&mut self, _before: &str, prefix: &str) -> Result { + fn predict(&mut self, context: &Context) -> Result { + let prefix = &context.prefix; let words: Vec = [ "water", "waffle", "walk", "WhatsApp", "café", "can't", "I'm", ] @@ -167,7 +176,7 @@ mod tests { use super::*; struct Fake(Result, ()>, Duration); impl Predict for Fake { - fn predict(&mut self, _before: &str, _prefix: &str) -> Result { + fn predict(&mut self, _: &Context) -> Result { std::thread::sleep(self.1); self.0.clone().map(|w| Prediction { words: super::super::model::slots(w), diff --git a/src-tauri/src/prediction/model.rs b/src-tauri/src/prediction/model.rs index 768c54e2..d6ec6274 100644 --- a/src-tauri/src/prediction/model.rs +++ b/src-tauri/src/prediction/model.rs @@ -1,5 +1,6 @@ //! English word prediction, with optional personal learning. Only the //! isolated worker holds context. +use super::context::Context; use sha2::{Digest, Sha256}; use std::{fs::File, io::Read, path::Path}; use switchify_prediction::{Options, Predictor}; @@ -22,13 +23,21 @@ pub trait Predict: Send { fn neural_status(&self) -> Option { None } - fn predict(&mut self, before: &str, prefix: &str) -> Result; + /// Statistical words for `context.before`; generation sees + /// `context.text`. + fn predict(&mut self, context: &Context) -> Result; fn poll(&mut self) -> Option { None } fn pending(&self) -> bool { false } + /// Drop pending results while the text continues, keeping the + /// generation context so it can be extended. + fn cancel(&mut self) { + self.reset(); + } + /// Drop pending results and the generation context. fn reset(&mut self) {} /// Learn one completed, non-overlapping piece of typed text. fn learn(&mut self, _text: &str) {} @@ -46,7 +55,7 @@ const GENERATED_WORDS: usize = 2; /// time, so the phrases are requested once the words have arrived. #[derive(Default)] struct Refining { - before: String, + text: String, prefix: String, words: Option, phrases: Option, @@ -178,7 +187,8 @@ impl Predict for Model { fn neural_status(&self) -> Option { self.neural.as_ref().map(|n| n.status()) } - fn predict(&mut self, before: &str, prefix: &str) -> Result { + fn predict(&mut self, context: &Context) -> Result { + let (before, prefix) = (context.before.as_str(), context.prefix.as_str()); // Child inference starts only after the parent has contained this worker // and sent a query, never during speculative statistical loading. if let Some(config) = self.config.take() { @@ -196,11 +206,12 @@ impl Predict for Model { .map(|s| s.word) .collect(); self.instant = std::array::from_fn(|i| words.get(i).cloned().map(display_word)); + // Generation also sees the earlier sentences. self.refining = Refining { - before: before.to_owned(), + text: context.text.clone(), prefix: prefix.to_owned(), words: self.neural.as_mut().and_then(|n| { - n.generate(before, prefix, 0, &words, GENERATED_WORDS) + n.generate(&context.text, prefix, 0, &words, GENERATED_WORDS) .ok() .flatten() }), @@ -219,13 +230,7 @@ impl Predict for Model { refining.generated = result.words; // Words first, then the phrases: a new request replaces any other. refining.phrases = neural - .generate_phrases( - &refining.before, - &refining.prefix, - 0, - &[], - PHRASE_SLOTS.len(), - ) + .generate_phrases(&refining.text, &refining.prefix, 0, &[], PHRASE_SLOTS.len()) .ok() .flatten(); Vec::new() @@ -249,6 +254,13 @@ impl Predict for Model { ) }) } + fn cancel(&mut self) { + self.refining = Refining::default(); + self.instant = Default::default(); + if let Some(n) = &mut self.neural { + n.cancel(); + } + } fn reset(&mut self) { self.refining = Refining::default(); self.instant = Default::default(); @@ -283,6 +295,47 @@ mod tests { let _ = fs::remove_dir_all(&self.0); } } + /// A context whose current sentence is all the text. + fn context(before: &str, prefix: &str) -> Context { + Context { + prefix: prefix.into(), + before: before.into(), + text: before.into(), + partial: false, + } + } + + #[test] + fn statistics_use_the_sentence_and_generation_the_earlier_text() { + let dir = Fixture::new(); + let path = dir.0.join("english.sqlite"); + switchify_prediction::build(&path, "I need help. I need help. Water.", "synthetic test") + .unwrap(); + let mut model = Model { + predictor: Predictor::open(&path, None).unwrap(), + neural: None, + config: None, + refining: Refining::default(), + instant: Default::default(), + learning: false, + }; + let earlier = Context { + text: "Water. I need ".into(), + ..context("I need ", "h") + }; + let words = model.predict(&earlier).unwrap().words; + assert_eq!( + words, + model.predict(&context("I need ", "h")).unwrap().words + ); + model.predict(&earlier).unwrap(); + // The phrases requested after the words see the same text. + assert_eq!(model.refining.text, "Water. I need "); + assert_eq!(model.refining.prefix, "h"); + model.cancel(); + assert!(model.refining.text.is_empty() && model.instant.iter().all(Option::is_none)); + } + #[test] fn generated_words_take_the_next_free_slot_and_phrases_their_own() { let instant = |words: &[&str]| std::array::from_fn(|i| words.get(i).map(|w| w.to_string())); @@ -341,7 +394,8 @@ mod tests { instant: Default::default(), learning, }; - let first = |model: &mut Model| model.predict("I need ", "h").unwrap().words[0].clone(); + let first = + |model: &mut Model| model.predict(&context("I need ", "h")).unwrap().words[0].clone(); let mut model = open(false); model.learn("I need hummus. I need hummus."); assert_eq!(first(&mut model).as_deref(), Some("help")); @@ -378,7 +432,7 @@ mod tests { threads: 4, }); assert_eq!( - model.predict("I need ", "h").unwrap().words[0] + model.predict(&context("I need ", "h")).unwrap().words[0] .as_deref() .unwrap(), "help" @@ -387,32 +441,32 @@ mod tests { assert!(model.config.is_none()); assert!(model.poll().is_none()); assert_eq!( - model.predict("I drink ", "").unwrap().words[0] + model.predict(&context("I drink ", "")).unwrap().words[0] .as_deref() .unwrap(), "water" ); assert!(model - .predict("", "zyzzy") + .predict(&context("", "zyzzy")) .unwrap() .words .iter() .all(Option::is_none)); assert_eq!( - model.predict("", "cafe\u{301}").unwrap().words, + model.predict(&context("", "cafe\u{301}")).unwrap().words, crate::prediction::slots(["café"]) ); assert_eq!( - model.predict("", "can’").unwrap().words, + model.predict(&context("", "can’")).unwrap().words, crate::prediction::slots(["can't"]) ); assert_eq!( - model.predict("", "i’").unwrap().words, + model.predict(&context("", "i’")).unwrap().words, crate::prediction::slots(["I'm"]) ); assert_eq!( model - .predict("", "h") + .predict(&context("", "h")) .unwrap() .words .iter() @@ -421,8 +475,8 @@ mod tests { 3 ); assert_eq!( - model.predict("I need.", "h").unwrap(), - model.predict("", "h").unwrap() + model.predict(&context("I need.", "h")).unwrap(), + model.predict(&context("", "h")).unwrap() ); drop(model); assert_eq!(fs::read(&path).unwrap(), original); @@ -486,7 +540,7 @@ mod tests { ("can you ", "h"), ][i % 4]; let start = Instant::now(); - let suggestions = model.predict(context, prefix).unwrap(); + let suggestions = model.predict(&self::context(context, prefix)).unwrap(); assert!(!suggestions.words.iter().all(Option::is_none)); if i >= 20 { samples.push(start.elapsed().as_secs_f64() * 1000.0); @@ -494,7 +548,7 @@ mod tests { } samples.sort_by(f64::total_cmp); assert_eq!( - model.predict("I need ", "he").unwrap().words[0] + model.predict(&context("I need ", "he")).unwrap().words[0] .as_deref() .unwrap(), "help" diff --git a/src-tauri/src/prediction/worker.rs b/src-tauri/src/prediction/worker.rs index 613849cc..f1617cad 100644 --- a/src-tauri/src/prediction/worker.rs +++ b/src-tauri/src/prediction/worker.rs @@ -343,7 +343,8 @@ impl Engine { None => return self.batch.clone(), } } else { - self.database.reset(); + // The text continues, so generation keeps its context. + self.database.cancel(); self.displayed = None; let (status, prediction) = self.database.predict(&ctx); self.status = status; @@ -435,7 +436,7 @@ impl Engine { return None; }; let insert = inserts.get(index)?.clone()?; - self.database.reset(); + self.database.cancel(); self.batch = None; self.displayed = None; self.inserts = Default::default(); @@ -454,7 +455,7 @@ impl Engine { sentence_start, } => { if self.generation != generation { - self.database.reset(); + self.database.cancel(); self.displayed = None; self.snapshot = None; } @@ -692,7 +693,10 @@ mod tests { pending: bool, } impl super::super::model::Predict for Deferred { - fn predict(&mut self, _: &str, _: &str) -> Result { + fn predict( + &mut self, + _: &super::super::context::Context, + ) -> Result { self.pending = true; Ok(super::super::model::Prediction { words: crate::prediction::slots(["water", "walk", "waffle"]), @@ -890,10 +894,79 @@ mod tests { ); } #[test] + fn generation_sees_earlier_sentences_and_keeps_its_context_while_typing() { + use std::sync::{Arc, Mutex}; + #[derive(Default)] + struct Seen { + contexts: Vec<(String, String, String)>, + cancels: usize, + resets: usize, + } + struct Recording(Arc>); + impl super::super::model::Predict for Recording { + fn predict( + &mut self, + context: &super::super::context::Context, + ) -> Result { + self.0.lock().unwrap().contexts.push(( + context.before.clone(), + context.text.clone(), + context.prefix.clone(), + )); + Ok(super::super::model::Prediction::default()) + } + fn cancel(&mut self) { + self.0.lock().unwrap().cancels += 1; + } + fn reset(&mut self) { + self.0.lock().unwrap().resets += 1; + } + } + let seen = Arc::new(Mutex::new(Seen::default())); + let mut e = engine(); + e.database = Database::with_predictor(Box::new(Recording(Arc::clone(&seen)))); + e.respond(request( + vec![append("We went to the beach. It was warm! The wa")], + 1, + None, + )); + // Starting to track the foreground clears the empty buffer once. + let initial = seen.lock().unwrap().resets; + e.respond(request(vec![append("v")], 2, None)); + e.respond(request(vec![append("es ")], 3, None)); + { + let seen = seen.lock().unwrap(); + let context = |before: &str, text: &str, prefix: &str| { + (before.to_owned(), text.to_owned(), prefix.to_owned()) + }; + // Statistics keep the current sentence; generation sees it all. + assert_eq!( + seen.contexts, + [ + context(" The ", "We went to the beach. It was warm! The ", "wa"), + context(" The ", "We went to the beach. It was warm! The ", "wav"), + context( + " The waves ", + "We went to the beach. It was warm! The waves ", + "" + ), + ] + ); + assert!(seen.cancels >= 3); + assert_eq!(seen.resets, initial); + } + // Enter clears the text and with it the generation context. + e.respond(request(vec![edit(Edit::End)], 4, None)); + assert!(seen.lock().unwrap().resets > initial); + } + #[test] fn unshown_words_leave_no_gap() { struct Gappy; impl super::super::model::Predict for Gappy { - fn predict(&mut self, _: &str, _: &str) -> Result { + fn predict( + &mut self, + _: &super::super::context::Context, + ) -> Result { Ok(super::super::model::Prediction { words: [ None, @@ -936,7 +1009,10 @@ mod tests { pending: bool, } impl super::super::model::Predict for Sparse { - fn predict(&mut self, _: &str, _: &str) -> Result { + fn predict( + &mut self, + _: &super::super::context::Context, + ) -> Result { self.pending = true; Ok(super::super::model::Prediction { words: crate::prediction::slots(["water"]), @@ -1554,7 +1630,10 @@ mod tests { } struct Phrasal(&'static str); impl super::super::model::Predict for Phrasal { - fn predict(&mut self, _: &str, _: &str) -> Result { + fn predict( + &mut self, + _: &super::super::context::Context, + ) -> Result { let mut words = crate::prediction::slots(["water"]); words[crate::prediction::PHRASE_SLOTS.start] = Some(self.0.into()); Ok(super::super::model::Prediction { words }) @@ -1724,7 +1803,10 @@ mod tests { struct Learner(std::sync::Arc>>); impl super::super::model::Predict for Learner { - fn predict(&mut self, _: &str, _: &str) -> Result { + fn predict( + &mut self, + _: &super::super::context::Context, + ) -> Result { Ok(Default::default()) } fn learn(&mut self, text: &str) {