Add offline word prediction to the scanning keyboard

.github/workflows/ci.yml

@@ -40,7 +40,7 @@ jobs:
       - name: Reject retired product identity
         run: |
           retired_identity="pre""view"
-          if git grep -n -i "$retired_identity" -- ':!src-tauri/Cargo.lock'; then
+          if git grep -I -n -i "$retired_identity" -- ':!src-tauri/Cargo.lock'; then
             exit 1
           fi
       - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6

docs/qwerty-keyboard.md

@@ -16,7 +16,7 @@ The bottom control row starts with Close keyboard, followed by Letters, Navigati
 
 Modifier keys are pressed only around each emitted shortcut and immediately released. Selecting a locked modifier does not hold that operating-system key while the scanner runs. Ordinary characters use text injection; command combinations and navigation use native key events. Native shortcuts retain the operating system's layout semantics.
 
-The keyboard closes when the foreground target or display environment changes, or its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. No typed text is logged, and no prediction data, context reader, protocol extension or persistent keyboard settings are added.
+The keyboard closes when the foreground target or display environment changes, or its scan session ends. Failed input clears keyboard modifiers and requires Select to resume. Stop, disconnect and application exit use the shared deterministic input cleanup. Word prediction can be enabled in Scanning settings. Five suggestions appear on the Letters page. Selecting a suggestion inserts its missing suffix and a space. Prediction uses local, read-only data; no personal vocabulary is saved. See [word prediction](word-prediction.md) for context availability and validation.
 
 ## Manual validation
 

docs/word-prediction.md

@@ -0,0 +1,42 @@
+# Word prediction
+
+Word prediction is enabled by default in Scanning settings. Its five-position row appears only on the Letters page. Empty positions are skipped. Select the row and then a word using the existing switches. Acceptance appends the missing suffix and a space without deleting text, selecting text or using the clipboard. Shift and Caps affect completion casing; Ctrl, Alt/Option and Windows/Command suppress suggestions.
+
+The bundled English SQLite database works offline. Ranking backs off from the longest available three-word context to shorter contexts, then to word frequency. Prefix matching is case-insensitive. Duplicate words are removed. The data is fixed; typing is never used to train or update it.
+
+Windows uses UI Automation and macOS uses Accessibility range attributes. Both read at most 512 preceding characters and a short following range, reject protected fields, and hide predictions for selections or mid-word insertion. Applications that do not expose reliable text ranges may have no predictions. No compatibility claim follows from compilation alone.
+
+If an editable non-protected field can be identified but its text is unavailable, the fallback remembers only successful Switchify typing after a word boundary entered through Switchify. It does not reconstruct existing text. Navigation, deletion, shortcuts, external typing/clicks, focus changes and failures clear this buffer. Fallback is unavailable when its passive activity observer cannot start. The observer records only an activity counter, never external keys or text. Numbers and navigation pages do not retain fallback typing.
+
+A separate process performs accessibility and database work. Private bounded inherited pipes carry results to native rendering. Text and suggestions are not sent to the React UI, diagnostic history or telemetry. One request is outstanding at a time with a two-second deadline. Timeout stops predictions until the keyboard is reopened; ordinary keyboard operation remains available. Closing the keyboard, ending scanning, or exiting kills and reaps the worker.
+
+Before a suggestion is accepted, the worker checks the field, protection state, text/selection snapshot and external activity again. Any mismatch rejects the action. Verification and native insertion cannot be atomic across applications; a field can still change in that short interval.
+
+## Validation
+
+Automated tests use in-memory databases, fake accessibility adapters and sleeping subprocesses. They never inject desktop input. The database benchmark can be run explicitly with:
+
+```powershell
+cargo test --manifest-path src-tauri/Cargo.toml bundled_database_benchmark -- --ignored --nocapture
+```
+
+This benchmark prints counts and timings only and does not read a focused field. Its ignored status keeps performance measurement separate from correctness checks.
+
+Native integration checks remain pending on Windows and macOS. The older standalone probe's Notepad result does not qualify this integration. Manually test synthetic text in Notepad, TextEdit, Edge/Chrome, Safari and Word where installed. Exercise typing, prediction acceptance, Shift/Caps, external edits, selection, caret movement, passwords, page changes, top/bottom docking, disconnect and app exit. On macOS use `npm run macos:run` to retain the signed Accessibility identity.
+
+For each app tested, collect at least 100 successful reads and report median, p95, maximum and failures. Target warm context-plus-query p95 below 50 ms; report startup and the 250-ms polling interval separately. Unsupported and untested apps must remain identified as such.
+
+Prepare a disposable field containing `I would like some wa`, with the caret after `wa`. From the repository root run the command below, then focus that field during its five-second countdown. It samples every 250 ms, stops after 100 supported warm samples (or 200 attempts), and prints only counts, fixture comparisons and timings. It sends read-only queries; it cannot accept a word or inject input. The first request includes worker/database startup and is excluded from warm percentiles.
+
+```powershell
+node scripts/measure-word-prediction.mjs src-tauri/target/debug/switchify-pc.exe Notepad water
+```
+
+For macOS, pass the signed app's `Contents/MacOS/switchify-pc` executable after building with `npm run macos:run`. Use a fresh run for each app. Keep all focused fields synthetic throughout measurement. Repeat functional checks with an empty field, `Hello. wa`, multiline text, `café naïve wa`, a pasted sentence, deletion, a caret inside a word, a nonempty selection, selection replacement, and switching between two fields. Verify that protected fields produce no suggestions and that closing the keyboard removes the worker. Do not count empty/unsupported responses as successful latency samples.
+
+Measured locally on Windows in a debug build (200 synthetic database queries): median 2.59 ms, p95 10.78 ms, maximum 14.35 ms, zero query failures. Database startup took 251.13 ms. These numbers exclude accessibility and IPC and are not app compatibility results.
+
+| App | Integration status |
+| --- | --- |
+| Notepad, Edge, Chrome, Word on Windows | Untested with this integration |
+| TextEdit, Safari, Chrome, Word on macOS | Untested; macOS host unavailable locally |

scripts/measure-word-prediction.mjs

@@ -0,0 +1,97 @@
+// Manual, read-only fixture measurement. Never sends an Accept request.
+import { spawn } from 'node:child_process';
+import { resolve } from 'node:path';
+import { setTimeout as delay } from 'node:timers/promises';
+import { performance } from 'node:perf_hooks';
+
+const [executable, app, expectedWord] = process.argv.slice(2);
+if (!executable || !app || !expectedWord) {
+  console.error('Usage: node scripts/measure-word-prediction.mjs <executable> <app-label> <expected-word>');
+  process.exit(1);
+}
+console.log('Focus a synthetic test field now. Sampling starts in five seconds. No input will be injected.');
+await delay(5000);
+const child = spawn(resolve(executable), [
+  '--switchify-prediction-worker',
+  resolve('src-tauri/resources/WordData2017051601.db'),
+  '[]',
+], { stdio: ['pipe', 'pipe', 'ignore'], windowsHide: true });
+let buffer = Buffer.alloc(0);
+let pending;
+let failed = false;
+const fail = () => {
+  failed = true;
+  pending?.reject(new Error('Worker unavailable'));
+  pending = undefined;
+};
+child.on('error', fail);
+child.on('exit', fail);
+child.stdin.on('error', fail);
+child.stdout.on('data', chunk => {
+  buffer = Buffer.concat([buffer, chunk]);
+  if (buffer.length < 4) return;
+  const length = buffer.readUInt32LE();
+  if (!length || length > 16384 || buffer.length > length + 4) return fail();
+  if (buffer.length !== length + 4) return;
+  try {
+    const response = JSON.parse(buffer.subarray(4).toString('utf8'));
+    buffer = Buffer.alloc(0);
+    const request = pending;
+    pending = undefined;
+    if (!request) return fail();
+    request.resolve(response);
+  } catch { fail(); }
+});
+async function query(generation) {
+  if (failed) throw new Error('Worker unavailable');
+  const bytes = Buffer.from(JSON.stringify({ Query: {
+    generation, edit: null, reset: true, shift: false, caps: false,
+  } }));
+  const header = Buffer.alloc(4);
+  header.writeUInt32LE(bytes.length);
+  let timer;
+  try {
+    return await new Promise((resolve, reject) => {
+      pending = { resolve, reject };
+      timer = setTimeout(() => { fail(); child.kill(); }, 2000);
+      child.stdin.write(Buffer.concat([header, bytes]));
+    });
+  } finally { clearTimeout(timer); }
+}
+const stop = () => { fail(); child.kill(); };
+process.on('SIGINT', stop);
+process.on('SIGTERM', stop);
+const times = [];
+let unsupported = 0;
+let failures = 0;
+let fixtureMatches = 0;
+let firstRequestMs;
+try {
+  // Up to 200 attempts, stopping after 100 supported samples.
+  for (let generation = 0; generation < 200 && times.length < 100; generation++) {
+    const start = performance.now();
+    const response = await query(generation);
+    const elapsed = performance.now() - start;
+    if (generation === 0) firstRequestMs = elapsed;
+    const result = response.Suggestions;
+    if (!result || result.generation !== generation) throw new Error('Invalid response');
+    if (result.batch?.words?.length) {
+      if (generation !== 0) times.push(elapsed);
+      if (result.batch.words.some(word => word.toLowerCase() === expectedWord.toLowerCase())) fixtureMatches++;
+    } else unsupported++;
+    await delay(Math.max(0, 250 - elapsed));
+  }
+} catch { failures++; }
+finally {
+  child.kill();
+  process.off('SIGINT', stop);
+  process.off('SIGTERM', stop);
+}
+times.sort((a, b) => a - b);
+const percentile = fraction => times.length ? +times[Math.ceil(times.length * fraction) - 1].toFixed(2) : null;
+console.log(JSON.stringify({
+  app, successfulSamples: times.length, unsupported, failures, fixtureMatches,
+  firstRequestMs: firstRequestMs === undefined ? null : +firstRequestMs.toFixed(2),
+  medianMs: percentile(0.5), p95Ms: percentile(0.95), maximumMs: percentile(1),
+}, null, 2));
+if (failures || times.length < 100) process.exitCode = 1;

src-tauri/Cargo.toml

@@ -21,6 +21,8 @@ enigo = "=0.6.1"
 hmac = "0.12"
 keyring = "4.1.6"
 rand = "0.8"
+rusqlite = { version = "0.40.2", features = ["bundled"] }
+unicode-segmentation = "1.12"
 reqwest = { version = "0.13", default-features = false, features = ["json", "rustls"] }
 serde = { version = "1", features = ["derive"] }
 serde_json = "1"
@@ -60,11 +62,15 @@ windows = { version = "0.62.2", features = [
   "Win32_Security",
   "Win32_Storage_FileSystem",
   "Win32_System_LibraryLoader",
+  "Win32_System_Com",
+  "Win32_System_Ole",
+  "Win32_System_Variant",
   "Win32_System_Power",
   "Win32_System_Registry",
   "Win32_System_Threading",
   "Win32_System_WinRT",
   "Win32_UI_HiDpi",
+  "Win32_UI_Accessibility",
   "Win32_UI_Input_KeyboardAndMouse",
   "Win32_UI_WindowsAndMessaging"
 ] }

src-tauri/resources/PROVENANCE.md

@@ -0,0 +1,10 @@
+# English prediction database
+
+`WordData2017051601.db` was recovered from the user's private `enaboapps/sayit-ios` history, path `Assets/Databases/WordData2017051601.db`, commit `95bda9265758e50787c5da2779b16f36b53c319d`, dated May 20, 2017. Recovery took place September 18, 2026. The user authorized bundling this database with Switchify.
+
+- Size: 102,690,816 bytes.
+- SHA-256: `dedd65d263bde8315e7e5ed7d2c8e04f17c33598a68506c9d70661a6d1f57318`.
+- SQLite integrity check during recovery: `ok`.
+- Rows: WORDS 165,420; BIGRAMS 976,804; TRIGRAMS 996,846; QUADGRAMS 953,630.
+
+This is the unchanged original SQLite file. The later Realm archive is not used. Runtime connections are read-only and ignore personal frequency columns. No captured text or user vocabulary is written into this resource.

src-tauri/src/lib.rs

@@ -20,6 +20,7 @@ mod point_scan;
 mod point_scan_activation;
 mod point_scan_runtime;
 mod point_workflow;
+mod prediction;
 mod protocol;
 mod remote_scan;
 mod scan_executor;
@@ -1361,6 +1362,9 @@ fn point_scan_prepare(app: &AppHandle) -> Result<(), String> {
 }
 
 pub fn run() {
+    if prediction::worker::run_from_args() {
+        return;
+    }
     #[cfg(target_os = "windows")]
     if switch_input::run_worker_from_args() {
         return;