From 383c95c48b0facc032c231649324feeecae85f58 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Sun, 27 Sep 2026 16:27:32 +0100 Subject: [PATCH 1/3] Hold suggestions until the next word after context is lost mid-word When the prediction context is cleared inside a word and the caret has not moved away, nothing is sent to the worker until a character that ends the word is typed. The main process tracks only whether each typed character was part of a word, never the text. Closes #926 Co-Authored-By: Claude Fable 5.1 --- docs/word-prediction.md | 6 +- src-tauri/src/prediction/mod.rs | 284 +++++++++++++++++++++++++++++--- src-tauri/src/scan_keyboard.rs | 5 + 3 files changed, 270 insertions(+), 25 deletions(-) diff --git a/docs/word-prediction.md b/docs/word-prediction.md index 42c23ff1..d3753021 100644 --- a/docs/word-prediction.md +++ b/docs/word-prediction.md @@ -6,13 +6,15 @@ Predictions use only a temporary buffer of successful Switchify keyboard input, The buffer holds at most 512 characters, retaining complete Unicode graphemes at its leading boundary. It tracks ordinary characters, spaces, Backspace and accepted completions across keyboard pages and top/bottom docking. Navigation, Delete, Enter, Tab, shortcuts, failed input, external typing/clicks/scrolling and foreground changes clear context. Opening or closing the keyboard, ending scanning, disconnecting and exiting also discard it, by killing the process that held it. The keyboard remains open across foreground changes, resets modifiers and suggestions, and sends subsequent input to the new foreground application. +When context is cleared inside a word and the caret has not moved away, the letters that finish that word would be completed as if they began a new one: after “hel”, a further “l” could be completed to “like” and leave “hellike”. Suggestions are therefore held until a space, punctuation or other character that ends the word has been typed, and resume with the next word. This applies after a key that failed to type, a navigation or Delete key, Retry predictions and a worker replaced after a missed deadline. Backspace does not release the hold, since it may uncover more of the word. Context cleared between words holds nothing. Nor does context cleared by a foreground change, a shortcut, Enter, Tab or outside keyboard and mouse activity, because typing then continues somewhere else; these also end a hold. To know whether a word is in progress, the main process remembers for each character Switchify typed only whether it was part of a word, never the character, for at most 512 characters. + A passive observer records only an activity counter and timestamp, never external text. Prediction is unavailable if this observer cannot start or loses access. Edits made before the observer is ready are discarded because intervening activity cannot be verified. Each queued edit is scoped to its foreground target and the time before injection, so edits preceding an observed external change are discarded. Changes within an application that produce no observed input cannot be detected without inspecting its fields. ## On-device model One Word prediction setting controls an offline SmolLM2-135M int8 ONNX model. The saved `enhancedWordPrediction` field is retained for compatibility but does not choose an engine. The model spells candidates from subword pieces and reads only the last 256 characters of the temporary buffer, beginning at a word boundary. It never learns from typing. -For a first typed prefix, a fixed local context keeps the model from favoring website names at the start of a document; it adds no user text. The model loads in the worker while keyboard input remains available, which takes about a second. Suggestions are blank until loading finishes. Later keyboard opens normally skip this wait by using the spare worker described under Worker process. A passive badge beside the scan prompt distinguishes loading, a ready keyboard awaiting typed context, no matching suggestions, available suggestions, paused activity tracking, and prediction failure. It never shows typed text and is not a scan target. If loading or inference fails, the keyboard continues accepting input and offers **Retry predictions** in its toolbar. A call that exceeds 1.5 seconds is tolerated: its suggestions are still shown, and only three such calls in a row count as a failure, since one stall on a busy machine does not mean the model cannot keep up. A call at normal speed clears the count. A call so slow that its reply misses the two-second deadline is handled by replacing the worker, described under Worker process. Retry restarts only the prediction worker, clears its private text context, and leaves the keyboard open; type a new prefix afterward. A 400 ms search budget bounds candidate exploration. A clipped buffer with no complete earlier word yields no suggestions. +For a first typed prefix, a fixed local context keeps the model from favoring website names at the start of a document; it adds no user text. The model loads in the worker while keyboard input remains available, which takes about a second. Suggestions are blank until loading finishes. Later keyboard opens normally skip this wait by using the spare worker described under Worker process. A passive badge beside the scan prompt distinguishes loading, a ready keyboard awaiting typed context, suggestions held until the next word, no matching suggestions, available suggestions, paused activity tracking, and prediction failure. It never shows typed text and is not a scan target. If loading or inference fails, the keyboard continues accepting input and offers **Retry predictions** in its toolbar. A call that exceeds 1.5 seconds is tolerated: its suggestions are still shown, and only three such calls in a row count as a failure, since one stall on a busy machine does not mean the model cannot keep up. A call at normal speed clears the count. A call so slow that its reply misses the two-second deadline is handled by replacing the worker, described under Worker process. Retry restarts only the prediction worker, clears its private text context, and leaves the keyboard open. Suggestions resume with the next word, or at once when no word was in progress. A 400 ms search budget bounds candidate exploration. A clipped buffer with no complete earlier word yields no suggestions. Candidates contain ASCII letters and apostrophes and must have sufficient model probability. There is no vocabulary filter: any word the model finds likely can be suggested, including swearing, because the person typing chose it. Single-letter candidates are limited to “a” and “I”. Up to five suggestions are shown: the first, third and fifth are the most likely single words, and the second and fourth are the two most likely two-word phrases that begin with one of the top three words, ranked by the probability of the pair. When fewer phrases are found within an extra 200 ms, single words fill the remaining slots, and the other way round. Accepting a phrase inserts the rest of its first word, a space, the second word and a trailing space. @@ -62,7 +64,7 @@ The model files are too large to commit. `npm run prediction-model` downloads th ## Worker process -A separate process owns the buffer, activity observer and model. Private bounded inherited pipes carry successful edits and results to native rendering. Text and suggestions are not sent to the React UI, diagnostic history or telemetry. One request is outstanding at a time with a two-second deadline. When a reply misses it, the worker is killed and reaped with its text context and a new worker starts by itself, so suggestions return after the model loads. As after any reset of the context, the new worker knows only what is typed from then on, so a word that was half typed is completed from its remaining letters alone; suggestions are reliable again from the next word. A second missed deadline within 60 seconds of that replacement stops prediction and offers Retry predictions, whatever the replacement answered in between. So does a third missed deadline while the same keyboard is open, however far apart they are. Reopening the keyboard or selecting Retry predictions starts the count again. An acceptance that was waiting on the missed reply fails and types nothing; the keyboard shows its error state for that acceptance while prediction itself recovers. Ordinary keyboard operation remains available throughout. Closing the keyboard, ending scanning, or exiting kills and reaps the worker. +A separate process owns the buffer, activity observer and model. Private bounded inherited pipes carry successful edits and results to native rendering. Text and suggestions are not sent to the React UI, diagnostic history or telemetry. One request is outstanding at a time with a two-second deadline. When a reply misses it, the worker is killed and reaped with its text context and a new worker starts by itself, so suggestions return after the model loads. If a word was in progress, suggestions resume with the next word. A second missed deadline within 60 seconds of that replacement stops prediction and offers Retry predictions, whatever the replacement answered in between. So does a third missed deadline while the same keyboard is open, however far apart they are. Reopening the keyboard or selecting Retry predictions starts the count again. An acceptance that was waiting on the missed reply fails and types nothing; the keyboard shows its error state for that acceptance while prediction itself recovers. Ordinary keyboard operation remains available throughout. Closing the keyboard, ending scanning, or exiting kills and reaps the worker. ### Spare worker diff --git a/src-tauri/src/prediction/mod.rs b/src-tauri/src/prediction/mod.rs index fd9f70c7..8dacbefc 100644 --- a/src-tauri/src/prediction/mod.rs +++ b/src-tauri/src/prediction/mod.rs @@ -180,6 +180,18 @@ struct Service { /// often since the keyboard opened. recovered: Option, recoveries: u8, + /// Whether each character Switchify typed belongs to a word, newest + /// last. It holds no text, and says if a word is in progress when the + /// context is lost. + shape: Vec, + /// Backspace went past the start of `shape`, into text never seen. + beyond: bool, + /// The context was lost inside a word, so its remaining letters would be + /// completed as if they began one. Nothing reaches the worker until a + /// word boundary is typed. + held: bool, + /// The activity epoch last seen, to notice input from outside. + epoch: Option, /// What the worker was started with, to start its replacement. model: Option, ignored: Vec, @@ -230,13 +242,74 @@ pub fn reset() { SERVICE.with(|s| { let mut s = s.borrow_mut(); s.generation = s.generation.wrapping_add(1); - s.edit.clear(); - s.reset = true; + s.moved(); s.last = None; s.accept = None; }); } +/// Letters, digits and apostrophes continue a word; anything else ends it. +fn word_char(c: char) -> bool { + c.is_alphanumeric() || c == '\'' || c == '’' +} impl Service { + fn in_word(&self) -> bool { + self.shape.last().copied().unwrap_or(self.beyond) + } + fn shaped(&mut self, edit: &Edit) { + match edit { + Edit::Append(text) => { + self.shape.extend(text.chars().map(word_char)); + self.beyond = false; + let extra = self.shape.len().saturating_sub(512); + self.shape.drain(..extra); + } + Edit::Backspace => { + if self.shape.pop().is_none() { + self.beyond = true; + } + } + Edit::Reset => {} + } + } + /// Input from outside Switchify may have moved the caret, so what was + /// typed before says nothing about where the next letter lands. + fn observe(&mut self, epoch: u64) { + if self.epoch.replace(epoch).is_some_and(|seen| seen != epoch) { + self.shape.clear(); + self.beyond = false; + self.held = false; + } + } + /// The context is lost and typing continues somewhere else: another + /// window, a new line or field, or wherever a shortcut led. + fn moved(&mut self) { + self.shape.clear(); + self.beyond = false; + self.held = false; + self.edit.clear(); + self.reset = true; + } + /// The context is lost and the caret is still at the text Switchify + /// typed. Inside a word, suggestions wait for the next one. + fn interrupt(&mut self) { + self.held |= self.in_word(); + self.shape.clear(); + self.beyond = false; + self.edit.clear(); + self.reset = true; + } + /// A successful edit. While suggestions are held it is not sent; the + /// first character that ends the word releases the hold. + fn typed(&mut self, edit: Edit, scope: InputScope) { + if !self.held { + self.queue_edit(edit, scope); + return; + } + self.shaped(&edit); + self.held = self.in_word() || matches!(edit, Edit::Backspace); + self.edit.clear(); + self.reset = true; + } fn response_timed_out(&self, now: Instant) -> bool { // Model loading is asynchronous; only inference uses the query deadline. let deadline = Duration::from_secs(2); @@ -244,12 +317,35 @@ impl Service { .is_some_and(|sent| now.duration_since(sent) >= deadline) } + /// One keyboard stroke. `unchanged` is whether the foreground window + /// stayed the same while it was typed. + fn stroke(&mut self, stroke: Stroke, success: bool, unchanged: bool, scope: InputScope) { + if !unchanged || stroke.shortcut() { + self.moved(); + return; + } + let edit = stroke + .character() + .map(|c| Edit::Append(c.to_string())) + .or_else(|| { + (stroke.key == Key::Named("Backspace") && !stroke.modifiers[0]) + .then_some(Edit::Backspace) + }); + match edit { + Some(edit) if success => self.typed(edit, scope), + // Enter and Tab start a new line or field. + None if matches!(stroke.key, Key::Named("Enter" | "Tab")) => self.moved(), + // A key that failed, or one that edits or moves within the text. + _ => self.interrupt(), + } + } fn queue_edit(&mut self, edit: Edit, scope: InputScope) { self.edit_revision = self.edit_revision.wrapping_add(1); if self.edit.len() >= 512 { self.edit.clear(); self.reset = true; } + self.shaped(&edit); self.edit.push(scope.record(edit)); } fn take_edits(&mut self) -> Vec { @@ -324,7 +420,7 @@ impl Service { self.recovered = Some(now); self.recoveries += 1; keyboard.prediction_loading = true; - self.reset = true; + self.interrupt(); self.generation = self.generation.wrapping_add(1); self.last = None; } @@ -428,28 +524,20 @@ fn close_with(start: impl FnOnce(&Path, Vec) -> Result) { /// Restart only prediction. The keyboard and its scan session remain open, /// while the old worker and its private text context are discarded. pub fn retry() { + let held = SERVICE.with(|s| { + let s = s.borrow(); + (s.held || s.in_word(), s.epoch) + }); stop(); + SERVICE.with(|s| (s.borrow_mut().held, s.borrow_mut().epoch) = held); } pub fn record(stroke: Stroke, success: bool, scope: InputScope) { SERVICE.with(|s| { let mut s = s.borrow_mut(); s.generation = s.generation.wrapping_add(1); s.last = None; - if success && scope.unchanged() && !stroke.shortcut() { - let edit = stroke - .character() - .map(|c| Edit::Append(c.to_string())) - .or_else(|| { - (stroke.key == Key::Named("Backspace") && !stroke.modifiers[0]) - .then_some(Edit::Backspace) - }); - if let Some(edit) = edit { - s.queue_edit(edit, scope); - return; - } - } - s.edit.clear(); - s.reset = true; + s.observe(activity::snapshot().0); + s.stroke(stroke, success, scope.unchanged(), scope); }); } pub fn record_punctuation(mark: char, removed_space: bool, success: bool, scope: InputScope) { @@ -457,14 +545,17 @@ pub fn record_punctuation(mark: char, removed_space: bool, success: bool, scope: let mut s = s.borrow_mut(); s.generation = s.generation.wrapping_add(1); s.last = None; - if success && scope.unchanged() { + s.observe(activity::snapshot().0); + let unchanged = scope.unchanged(); + if success && unchanged { if removed_space { - s.queue_edit(Edit::Backspace, scope); + s.typed(Edit::Backspace, scope); } - s.queue_edit(Edit::Append(format!("{mark} ")), scope); + s.typed(Edit::Append(format!("{mark} ")), scope); + } else if unchanged { + s.interrupt(); } else { - s.edit.clear(); - s.reset = true; + s.moved(); } }); } @@ -529,6 +620,8 @@ pub fn poll(app: &AppHandle, keyboard: Option<&mut Keyboard>, enabled: bool, ign }; SERVICE.with(|slot| { let mut s = slot.borrow_mut(); + s.observe(activity::snapshot().0); + keyboard.prediction_held = s.held; let case = ( keyboard.prediction_shift(), keyboard.caps, @@ -886,6 +979,151 @@ mod tests { assert!(service.client.is_none()); } } + fn key(c: char) -> Stroke { + Stroke { + key: Key::Character(c, c.to_ascii_uppercase()), + modifiers: [false; 4], + caps: false, + } + } + fn named(name: &'static str) -> Stroke { + Stroke { + key: Key::Named(name), + ..key('a') + } + } + fn service(text: &str) -> Service { + let mut service = Service::default(); + for c in text.chars() { + service.stroke(key(c), true, true, InputScope::capture()); + } + service + } + fn sent(service: &mut Service) -> String { + service + .take_edits() + .into_iter() + .map(|e| match e.edit { + Edit::Append(text) => text, + Edit::Backspace => "<".into(), + Edit::Reset => "|".into(), + }) + .collect() + } + #[test] + fn context_lost_inside_a_word_holds_suggestions_until_the_next_word() { + let scope = InputScope::capture(); + let mut s = service("hel"); + s.interrupt(); + assert!(s.held); + for c in "lo".chars() { + s.stroke(key(c), true, true, scope); + assert!(s.held && s.edit.is_empty() && s.reset); + } + // Backspace may uncover more of the word, so it releases nothing. + s.stroke(named("Backspace"), true, true, scope); + assert!(s.held); + s.stroke(key(' '), true, true, scope); + assert!(!s.held && s.edit.is_empty()); + for c in "wa".chars() { + s.stroke(key(c), true, true, scope); + } + assert_eq!(sent(&mut s), "|wa"); + + // Punctuation ends the word too, with the space it removes ignored. + let mut s = service("hel"); + s.interrupt(); + s.typed(Edit::Backspace, scope); + s.typed(Edit::Append(". ".into()), scope); + assert!(!s.held && s.edit.is_empty()); + } + #[test] + fn context_lost_between_words_holds_nothing() { + let scope = InputScope::capture(); + for text in ["", "hello ", "hello. ", "hello\n"] { + let mut s = service(text); + s.interrupt(); + assert!(!s.held, "{text:?}"); + s.stroke(key('w'), true, true, scope); + assert_eq!(sent(&mut s), "|w"); + } + // Deleting back into a word, typed or already there, is inside one. + for (text, deleted) in [("hello ", 1), ("", 1), ("a ", 3)] { + let mut s = service(text); + for _ in 0..deleted { + s.stroke(named("Backspace"), true, true, scope); + } + s.interrupt(); + assert!(s.held, "{text:?}"); + } + } + #[test] + fn only_a_loss_that_leaves_the_caret_in_place_holds() { + let scope = InputScope::capture(); + // A failed key, a key within the text, and a replaced worker. + let mut s = service("hel"); + s.stroke(key('l'), false, true, scope); + assert!(s.held && s.reset); + let mut s = service("hel"); + s.stroke(named("ArrowLeft"), true, true, scope); + assert!(s.held); + let mut s = service("hel"); + s.timed_out(&mut Keyboard::new(false), Instant::now()); + assert!(s.held && !s.failed); + // Another window, a shortcut, a new line and outside input do not. + let mut s = service("hel"); + s.stroke(key('l'), true, false, scope); + assert!(!s.held && s.reset && !s.in_word()); + let mut s = service("hel"); + s.stroke( + Stroke { + modifiers: [false, true, false, false], + ..key('a') + }, + true, + true, + scope, + ); + assert!(!s.held && s.reset); + let mut s = service("hel"); + s.stroke(named("Enter"), true, true, scope); + assert!(!s.held && s.reset); + let mut s = service("hel"); + s.observe(4); + s.observe(4); + assert!(s.in_word()); + s.observe(5); + s.interrupt(); + assert!(!s.held); + // Outside input also ends a hold: the caret may be anywhere now. + let mut s = service("hel"); + s.observe(4); + s.interrupt(); + s.observe(5); + assert!(!s.held); + } + #[test] + fn retry_keeps_the_hold_and_discards_everything_else() { + stop(); + SERVICE.with(|slot| *slot.borrow_mut() = service("hel")); + retry(); + SERVICE.with(|slot| { + let s = slot.borrow(); + assert!(s.held && s.edit.is_empty() && s.shape.is_empty()); + assert_eq!(s.generation, 0); + }); + retry(); + SERVICE.with(|slot| assert!(slot.borrow().held)); + stop(); + SERVICE.with(|slot| assert!(!slot.borrow().held)); + } + #[test] + fn the_word_shape_holds_no_text_and_stays_bounded() { + let mut s = service("it's 4pm"); + assert_eq!(s.shape, [true, true, true, true, false, true, true, true]); + s.queue_edit(Edit::Append("a ".repeat(400)), InputScope::capture()); + assert_eq!(s.shape.len(), 512); + } #[test] fn invalid_edit_acknowledgements_fail_closed() { let mut service = Service { diff --git a/src-tauri/src/scan_keyboard.rs b/src-tauri/src/scan_keyboard.rs index 6c041d71..ca80c133 100644 --- a/src-tauri/src/scan_keyboard.rs +++ b/src-tauri/src/scan_keyboard.rs @@ -205,6 +205,8 @@ pub struct Keyboard { prediction_enabled: bool, prediction_failed: bool, pub(crate) prediction_loading: bool, + /// Suggestions wait for the next word after context was lost inside one. + pub(crate) prediction_held: bool, prediction_tracking: bool, predictions: Option, queued_predictions: Option, @@ -239,6 +241,7 @@ impl Keyboard { prediction_enabled: false, prediction_failed: false, prediction_loading: false, + prediction_held: false, prediction_tracking: true, predictions: None, queued_predictions: None, @@ -348,6 +351,8 @@ impl Keyboard { "Prediction loading".into() } else if !self.prediction_tracking { "Prediction paused".into() + } else if self.prediction_held { + "Suggestions resume next word".into() } else if self.queued_predictions.is_some() { "Suggestions updating".into() } else { From eb5e37d0a1387f6e95d52822549f70f7317892ac Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Sun, 27 Sep 2026 16:35:48 +0100 Subject: [PATCH 2/3] Close the remaining ways to complete a word fragment Review follow-up: a failed acceptance, a failed Enter, Tab or shortcut, and a shortcut inside a word now hold. Navigation keys always hold. Backspace into text the worker never had holds, and Backspace back to a word boundary releases. An observer that is not running no longer ends a hold. Co-Authored-By: Claude Fable 5.1 --- docs/word-prediction.md | 13 +- src-tauri/src/prediction/mod.rs | 257 ++++++++++++++++++++++---------- src-tauri/src/scan_keyboard.rs | 21 +++ 3 files changed, 210 insertions(+), 81 deletions(-) diff --git a/docs/word-prediction.md b/docs/word-prediction.md index d3753021..4266f1a3 100644 --- a/docs/word-prediction.md +++ b/docs/word-prediction.md @@ -6,7 +6,18 @@ Predictions use only a temporary buffer of successful Switchify keyboard input, The buffer holds at most 512 characters, retaining complete Unicode graphemes at its leading boundary. It tracks ordinary characters, spaces, Backspace and accepted completions across keyboard pages and top/bottom docking. Navigation, Delete, Enter, Tab, shortcuts, failed input, external typing/clicks/scrolling and foreground changes clear context. Opening or closing the keyboard, ending scanning, disconnecting and exiting also discard it, by killing the process that held it. The keyboard remains open across foreground changes, resets modifiers and suggestions, and sends subsequent input to the new foreground application. -When context is cleared inside a word and the caret has not moved away, the letters that finish that word would be completed as if they began a new one: after “hel”, a further “l” could be completed to “like” and leave “hellike”. Suggestions are therefore held until a space, punctuation or other character that ends the word has been typed, and resume with the next word. This applies after a key that failed to type, a navigation or Delete key, Retry predictions and a worker replaced after a missed deadline. Backspace does not release the hold, since it may uncover more of the word. Context cleared between words holds nothing. Nor does context cleared by a foreground change, a shortcut, Enter, Tab or outside keyboard and mouse activity, because typing then continues somewhere else; these also end a hold. To know whether a word is in progress, the main process remembers for each character Switchify typed only whether it was part of a word, never the character, for at most 512 characters. +When context is cleared inside a word and the caret has not moved away, the letters that finish that word would be completed as if they began a new one: after “hel”, a further “l” could be completed to “like” and leave “hellike”. Suggestions are therefore held until a space, punctuation or other character that ends the word has been typed, and resume with the next word. The badge reads “Suggestions resume next word” meanwhile. + +| Context is cleared by | Suggestions are held | +|---|---| +| A key that failed to type, a failed acceptance, Retry predictions, or a worker replaced after a missed deadline | When a word is in progress | +| A shortcut | When a word is in progress | +| An arrow key, Home, End, Page Up, Page Down, Delete or another key that is not typed text | Always, because the caret is then in text Switchify never saw and is taken to be inside a word | +| Backspace that deletes text the worker does not have, such as text that was there before the keyboard opened | When the character before the caret is part of a word or is unknown | +| Enter or Tab | Never; a new line or field begins | +| A foreground change, or keyboard and mouse activity from outside Switchify | Never; typing continues somewhere else. These also end a hold | + +Backspace releases a hold once it has deleted back to a space or punctuation Switchify typed. A hold costs at most the suggestions for one word; it never changes what is typed. To know whether a word is in progress, the main process remembers for each character Switchify typed only whether it was part of a word, never the character, for at most 512 characters. While the activity observer is not running, outside activity cannot be seen and does not end a hold. A passive observer records only an activity counter and timestamp, never external text. Prediction is unavailable if this observer cannot start or loses access. Edits made before the observer is ready are discarded because intervening activity cannot be verified. Each queued edit is scoped to its foreground target and the time before injection, so edits preceding an observed external change are discarded. Changes within an application that produce no observed input cannot be detected without inspecting its fields. diff --git a/src-tauri/src/prediction/mod.rs b/src-tauri/src/prediction/mod.rs index 8dacbefc..e4e118e8 100644 --- a/src-tauri/src/prediction/mod.rs +++ b/src-tauri/src/prediction/mod.rs @@ -192,6 +192,8 @@ struct Service { held: bool, /// The activity epoch last seen, to notice input from outside. epoch: Option, + /// Characters the worker has had since its context was last cleared. + known: usize, /// What the worker was started with, to start its replacement. model: Option, ignored: Vec, @@ -271,44 +273,64 @@ impl Service { Edit::Reset => {} } } + fn forget(&mut self) { + self.known = 0; + self.edit.clear(); + self.reset = true; + } /// Input from outside Switchify may have moved the caret, so what was - /// typed before says nothing about where the next letter lands. - fn observe(&mut self, epoch: u64) { - if self.epoch.replace(epoch).is_some_and(|seen| seen != epoch) { + /// typed before says nothing about where the next letter lands. An + /// observer that is not running cannot tell, and changes nothing. + fn observe(&mut self, (epoch, healthy): (u64, bool)) { + if healthy && self.epoch.replace(epoch).is_some_and(|seen| seen != epoch) { self.shape.clear(); self.beyond = false; self.held = false; + // The worker saw it too and cleared its own context. + self.known = 0; } } /// The context is lost and typing continues somewhere else: another - /// window, a new line or field, or wherever a shortcut led. + /// window, or a new line or field. fn moved(&mut self) { self.shape.clear(); self.beyond = false; self.held = false; - self.edit.clear(); - self.reset = true; + self.forget(); } - /// The context is lost and the caret is still at the text Switchify - /// typed. Inside a word, suggestions wait for the next one. + /// The context is lost and the caret has not moved. Inside a word, + /// suggestions wait for the next one. fn interrupt(&mut self) { self.held |= self.in_word(); + self.forget(); + } + /// The context is lost and the caret is somewhere in the text nearby, + /// which was never seen. After a navigation key it is taken to be + /// inside a word; after a shortcut, only if it was before. + fn displaced(&mut self, navigated: bool) { + self.held |= navigated || self.in_word(); self.shape.clear(); - self.beyond = false; - self.edit.clear(); - self.reset = true; + self.beyond = self.held; + self.forget(); } - /// A successful edit. While suggestions are held it is not sent; the - /// first character that ends the word releases the hold. + /// A successful edit. It is sent to the worker unless suggestions are + /// held, or it deletes text the worker never had and leaves the caret + /// inside a word. A character that ends the word, or a Backspace that + /// reaches one, releases the hold. fn typed(&mut self, edit: Edit, scope: InputScope) { - if !self.held { - self.queue_edit(edit, scope); + let unknown = matches!(edit, Edit::Backspace) && self.known == 0; + self.shaped(&edit); + if self.held || unknown { + self.held = self.in_word(); + self.forget(); return; } - self.shaped(&edit); - self.held = self.in_word() || matches!(edit, Edit::Backspace); - self.edit.clear(); - self.reset = true; + match &edit { + Edit::Append(text) => self.known += text.chars().count(), + Edit::Backspace => self.known -= 1, + Edit::Reset => {} + } + self.queue_edit(edit, scope); } fn response_timed_out(&self, now: Instant) -> bool { // Model loading is asynchronous; only inference uses the query deadline. @@ -320,10 +342,19 @@ impl Service { /// One keyboard stroke. `unchanged` is whether the foreground window /// stayed the same while it was typed. fn stroke(&mut self, stroke: Stroke, success: bool, unchanged: bool, scope: InputScope) { - if !unchanged || stroke.shortcut() { + if !unchanged { self.moved(); return; } + // A key that failed to type moved nothing, whatever it was. + if !success { + self.interrupt(); + return; + } + if stroke.shortcut() { + self.displaced(false); + return; + } let edit = stroke .character() .map(|c| Edit::Append(c.to_string())) @@ -332,11 +363,10 @@ impl Service { .then_some(Edit::Backspace) }); match edit { - Some(edit) if success => self.typed(edit, scope), + Some(edit) => self.typed(edit, scope), // Enter and Tab start a new line or field. None if matches!(stroke.key, Key::Named("Enter" | "Tab")) => self.moved(), - // A key that failed, or one that edits or moves within the text. - _ => self.interrupt(), + None => self.displaced(true), } } fn queue_edit(&mut self, edit: Edit, scope: InputScope) { @@ -345,7 +375,6 @@ impl Service { self.edit.clear(); self.reset = true; } - self.shaped(&edit); self.edit.push(scope.record(edit)); } fn take_edits(&mut self) -> Vec { @@ -376,8 +405,7 @@ impl Service { } else { self.tracking = tracking; if !tracking { - self.edit.clear(); - self.reset = true; + self.forget(); } } } @@ -391,6 +419,7 @@ impl Service { self.tracking = tracking; if !tracking { self.edit.clear(); + self.known = 0; } let visible = keyboard.page == Page::Letters && !keyboard.modifiers[1..].iter().any(|m| *m != Modifier::Off); @@ -524,19 +553,23 @@ fn close_with(start: impl FnOnce(&Path, Vec) -> Result) { /// Restart only prediction. The keyboard and its scan session remain open, /// while the old worker and its private text context are discarded. pub fn retry() { - let held = SERVICE.with(|s| { - let s = s.borrow(); - (s.held || s.in_word(), s.epoch) + let kept = SERVICE.with(|s| { + let mut s = s.borrow_mut(); + s.interrupt(); + (std::mem::take(&mut s.shape), s.beyond, s.held, s.epoch) }); stop(); - SERVICE.with(|s| (s.borrow_mut().held, s.borrow_mut().epoch) = held); + SERVICE.with(|s| { + let mut s = s.borrow_mut(); + (s.shape, s.beyond, s.held, s.epoch) = kept; + }); } pub fn record(stroke: Stroke, success: bool, scope: InputScope) { SERVICE.with(|s| { let mut s = s.borrow_mut(); s.generation = s.generation.wrapping_add(1); s.last = None; - s.observe(activity::snapshot().0); + s.observe(activity::snapshot()); s.stroke(stroke, success, scope.unchanged(), scope); }); } @@ -545,7 +578,7 @@ pub fn record_punctuation(mark: char, removed_space: bool, success: bool, scope: let mut s = s.borrow_mut(); s.generation = s.generation.wrapping_add(1); s.last = None; - s.observe(activity::snapshot().0); + s.observe(activity::snapshot()); let unchanged = scope.unchanged(); if success && unchanged { if removed_space { @@ -620,7 +653,7 @@ pub fn poll(app: &AppHandle, keyboard: Option<&mut Keyboard>, enabled: bool, ign }; SERVICE.with(|slot| { let mut s = slot.borrow_mut(); - s.observe(activity::snapshot().0); + s.observe(activity::snapshot()); keyboard.prediction_held = s.held; let case = ( keyboard.prediction_shift(), @@ -706,9 +739,9 @@ pub fn poll(app: &AppHandle, keyboard: Option<&mut Keyboard>, enabled: bool, ign let trailing_space = text.ends_with(' '); let contains_letter = text.chars().any(char::is_alphabetic); for _ in 0..backspaces { - s.queue_edit(Edit::Backspace, scope); + s.typed(Edit::Backspace, scope); } - s.queue_edit(Edit::Append(text), scope); + s.typed(Edit::Append(text), scope); Ok((trailing_space, contains_letter)) }) } else { @@ -726,8 +759,7 @@ pub fn poll(app: &AppHandle, keyboard: Option<&mut Keyboard>, enabled: bool, ign ); } } else { - s.edit.clear(); - s.reset = true; + s.interrupt(); let _ = crate::scan_executor::cleanup(); keyboard.failed(); } @@ -822,6 +854,11 @@ mod tests { #[test] fn punctuation_records_the_backspace_and_insert_as_one_successful_edit() { stop(); + SERVICE.with(|slot| { + let mut service = slot.borrow_mut(); + service.typed(Edit::Append(" ".into()), InputScope::capture()); + service.edit.clear(); + }); record_punctuation('.', true, true, InputScope::capture()); SERVICE.with(|slot| { let service = slot.borrow(); @@ -1020,7 +1057,6 @@ mod tests { s.stroke(key(c), true, true, scope); assert!(s.held && s.edit.is_empty() && s.reset); } - // Backspace may uncover more of the word, so it releases nothing. s.stroke(named("Backspace"), true, true, scope); assert!(s.held); s.stroke(key(' '), true, true, scope); @@ -1038,6 +1074,46 @@ mod tests { assert!(!s.held && s.edit.is_empty()); } #[test] + fn deleting_the_rest_of_the_word_releases_the_hold() { + let scope = InputScope::capture(); + let mut s = service("say hel"); + s.interrupt(); + for _ in 0..2 { + s.stroke(named("Backspace"), true, true, scope); + assert!(s.held); + } + s.stroke(named("Backspace"), true, true, scope); + assert!(!s.held); + for c in "hello".chars() { + s.stroke(key(c), true, true, scope); + } + assert_eq!(sent(&mut s), "|hello"); + } + #[test] + fn deleting_into_text_the_worker_never_had_holds() { + let scope = InputScope::capture(); + // Released by a space, which is then deleted: the caret is back + // inside the word whose start the worker never saw. + let mut s = service("hel"); + s.interrupt(); + s.stroke(key(' '), true, true, scope); + assert!(!s.held); + s.stroke(named("Backspace"), true, true, scope); + assert!(s.held); + s.stroke(key('l'), true, true, scope); + assert_eq!(sent(&mut s), "|"); + // Text that was there before the keyboard opened. + let mut s = Service::default(); + s.stroke(named("Backspace"), true, true, scope); + assert!(s.held); + // Deleting what the worker has is an ordinary edit. + let mut s = service("hello "); + s.stroke(named("Backspace"), true, true, scope); + s.stroke(named("Backspace"), true, true, scope); + assert!(!s.held); + assert_eq!(sent(&mut s), "hello <<"); + } + #[test] fn context_lost_between_words_holds_nothing() { let scope = InputScope::capture(); for text in ["", "hello ", "hello. ", "hello\n"] { @@ -1047,81 +1123,102 @@ mod tests { s.stroke(key('w'), true, true, scope); assert_eq!(sent(&mut s), "|w"); } - // Deleting back into a word, typed or already there, is inside one. - for (text, deleted) in [("hello ", 1), ("", 1), ("a ", 3)] { - let mut s = service(text); - for _ in 0..deleted { - s.stroke(named("Backspace"), true, true, scope); - } - s.interrupt(); - assert!(s.held, "{text:?}"); - } } #[test] - fn only_a_loss_that_leaves_the_caret_in_place_holds() { + fn only_a_loss_that_leaves_the_caret_nearby_holds() { let scope = InputScope::capture(); - // A failed key, a key within the text, and a replaced worker. - let mut s = service("hel"); - s.stroke(key('l'), false, true, scope); - assert!(s.held && s.reset); - let mut s = service("hel"); - s.stroke(named("ArrowLeft"), true, true, scope); - assert!(s.held); + let shortcut = |key: Stroke| Stroke { + modifiers: [false, true, false, false], + ..key + }; + // A key that failed, whatever it was, and a replaced worker. + for failed in [key('l'), named("Enter"), named("Tab"), shortcut(key('s'))] { + let mut s = service("hel"); + s.stroke(failed, false, true, scope); + assert!(s.held && s.reset, "{failed:?}"); + } let mut s = service("hel"); s.timed_out(&mut Keyboard::new(false), Instant::now()); assert!(s.held && !s.failed); - // Another window, a shortcut, a new line and outside input do not. + // A shortcut holds inside a word only. let mut s = service("hel"); - s.stroke(key('l'), true, false, scope); - assert!(!s.held && s.reset && !s.in_word()); - let mut s = service("hel"); - s.stroke( - Stroke { - modifiers: [false, true, false, false], - ..key('a') - }, - true, - true, - scope, - ); + s.stroke(shortcut(key('s')), true, true, scope); + assert!(s.held); + let mut s = service("hello "); + s.stroke(shortcut(key('a')), true, true, scope); assert!(!s.held && s.reset); + // A navigation key lands in text never seen, so it always holds. + for name in ["ArrowLeft", "ArrowUp", "Home", "End", "Delete"] { + let mut s = service("hello "); + s.stroke(named(name), true, true, scope); + assert!(s.held, "{name}"); + s.stroke(key(' '), true, true, scope); + assert!(!s.held); + } + // Another window, a new line and outside input hold nothing. let mut s = service("hel"); - s.stroke(named("Enter"), true, true, scope); - assert!(!s.held && s.reset); + s.stroke(key('l'), true, false, scope); + assert!(!s.held && s.reset && !s.in_word()); + for name in ["Enter", "Tab"] { + let mut s = service("hel"); + s.stroke(named(name), true, true, scope); + assert!(!s.held && s.reset); + } let mut s = service("hel"); - s.observe(4); - s.observe(4); + s.observe((4, true)); + s.observe((4, true)); assert!(s.in_word()); - s.observe(5); + s.observe((5, true)); s.interrupt(); assert!(!s.held); // Outside input also ends a hold: the caret may be anywhere now. let mut s = service("hel"); - s.observe(4); + s.observe((4, true)); s.interrupt(); - s.observe(5); + // An observer that is not running cannot say anything happened. + s.observe((5, false)); + assert!(s.held); + s.observe((5, true)); assert!(!s.held); } #[test] + fn failed_punctuation_holds_inside_a_word() { + stop(); + for c in "hel".chars() { + record(key(c), true, InputScope::capture()); + } + record_punctuation('.', false, false, InputScope::capture()); + SERVICE.with(|slot| { + let s = slot.borrow(); + // Outside input during the test would end the hold, not start one. + assert!(s.held || s.shape.is_empty()); + assert!(s.edit.is_empty() && s.reset); + }); + stop(); + } + #[test] fn retry_keeps_the_hold_and_discards_everything_else() { stop(); - SERVICE.with(|slot| *slot.borrow_mut() = service("hel")); + SERVICE.with(|slot| *slot.borrow_mut() = service("say hel")); retry(); SERVICE.with(|slot| { let s = slot.borrow(); - assert!(s.held && s.edit.is_empty() && s.shape.is_empty()); - assert_eq!(s.generation, 0); + assert!(s.held && s.edit.is_empty()); + assert_eq!((s.generation, s.known, s.shape.len()), (0, 0, 7)); }); retry(); SERVICE.with(|slot| assert!(slot.borrow().held)); stop(); - SERVICE.with(|slot| assert!(!slot.borrow().held)); + SERVICE.with(|slot| { + let s = slot.borrow(); + assert!(!s.held && s.shape.is_empty()); + }); } #[test] fn the_word_shape_holds_no_text_and_stays_bounded() { let mut s = service("it's 4pm"); assert_eq!(s.shape, [true, true, true, true, false, true, true, true]); - s.queue_edit(Edit::Append("a ".repeat(400)), InputScope::capture()); + s.typed(Edit::Append("a ".repeat(400)), InputScope::capture()); assert_eq!(s.shape.len(), 512); } #[test] diff --git a/src-tauri/src/scan_keyboard.rs b/src-tauri/src/scan_keyboard.rs index ca80c133..24e23c26 100644 --- a/src-tauri/src/scan_keyboard.rs +++ b/src-tauri/src/scan_keyboard.rs @@ -1651,6 +1651,27 @@ mod tests { } } #[test] + fn a_hold_is_explained_unless_something_more_pressing_is() { + let mut k = Keyboard::new(true); + k.enable_predictions(true); + k.predictions(None, false); + k.prediction_held = true; + assert_eq!( + k.prediction_note().as_deref(), + Some("Suggestions resume next word") + ); + k.prediction_loading = true; + assert_eq!(k.prediction_note().as_deref(), Some("Prediction loading")); + k.predictions(None, true); + assert_eq!( + k.prediction_note().as_deref(), + Some("Prediction unavailable") + ); + k.predictions(None, false); + k.prediction_held = false; + assert_eq!(k.prediction_note().as_deref(), Some("Type for suggestions")); + } + #[test] fn prediction_warning_does_not_hide_resume_or_error_instructions() { let mut k = Keyboard::new(false); k.prediction_failed = true; From 18c8a639c12a4c797efe7c595d11283d61c6d834 Mon Sep 17 00:00:00 2001 From: Owen McGirr Date: Sun, 27 Sep 2026 16:41:23 +0100 Subject: [PATCH 3/3] Remember what lies under the typed text across a space Review follow-up: a space typed and deleted again after navigation or a Backspace into unseen text holds again. The observer starting again no longer ends a hold, the failed punctuation test asserts the hold, and the native validation steps describe the held cases. Co-Authored-By: Claude Fable 5.1 --- docs/word-prediction.md | 4 +-- src-tauri/src/prediction/mod.rs | 49 ++++++++++++++++++++++++--------- 2 files changed, 38 insertions(+), 15 deletions(-) diff --git a/docs/word-prediction.md b/docs/word-prediction.md index 4266f1a3..672c0522 100644 --- a/docs/word-prediction.md +++ b/docs/word-prediction.md @@ -17,7 +17,7 @@ When context is cleared inside a word and the caret has not moved away, the lett | Enter or Tab | Never; a new line or field begins | | A foreground change, or keyboard and mouse activity from outside Switchify | Never; typing continues somewhere else. These also end a hold | -Backspace releases a hold once it has deleted back to a space or punctuation Switchify typed. A hold costs at most the suggestions for one word; it never changes what is typed. To know whether a word is in progress, the main process remembers for each character Switchify typed only whether it was part of a word, never the character, for at most 512 characters. While the activity observer is not running, outside activity cannot be seen and does not end a hold. +Backspace releases a hold once it has deleted back to a space or punctuation Switchify typed. A hold costs at most the suggestions for one word; it never changes what is typed. To know whether a word is in progress, the main process remembers for each character Switchify typed only whether it was part of a word, never the character, for at most 512 characters. While the activity observer is not running, outside activity cannot be seen and does not end a hold, and neither does the observer starting again. Changing the switch key assignments counts as outside activity. A passive observer records only an activity counter and timestamp, never external text. Prediction is unavailable if this observer cannot start or loses access. Edits made before the observer is ready are discarded because intervening activity cannot be verified. Each queued edit is scoped to its foreground target and the time before injection, so edits preceding an observed external change are discarded. Changes within an application that produce no observed input cannot be detected without inspecting its fields. @@ -98,7 +98,7 @@ For native validation, use disposable synthetic text in Notepad and a browser on 1. Open the keyboard from the action menu and an assigned Open keyboard switch, including from idle, a paused scan and an active drag. Verify no click or focus change occurs and owned drag input is released. 2. Type `w`, then `a`, accept `water`, and continue typing. Existing or pasted text must never become prediction context. 3. Change pages and docking, type punctuation and numbers, and return to Letters. Verify context survives these layout changes and Backspace edits the tracked buffer. -4. Use navigation, shortcuts, failed edits and external keyboard/mouse activity. Verify suggestions clear and a new first letter starts fresh context. +4. Use external keyboard/mouse activity. Verify suggestions clear and a new first letter starts fresh context. Then type `hel`, press an arrow key, and type `l`: verify no suggestions appear and the badge reads “Suggestions resume next word” until a space is typed. Repeat with Retry predictions and with a shortcut in place of the arrow key. 5. Change foreground apps with locked modifiers selected. Verify the keyboard stays open, modifiers and suggestions reset, and later keys go to the new app. 6. Close the keyboard, stop scanning, disconnect and exit. Verify input releases and the prediction worker exits. After closing the keyboard one spare worker remains; reopen after a few seconds and within two minutes, and verify the loading badge clears almost immediately, then verify the spare exits after two minutes idle and when scanning stops. Test observer failure separately; typing should remain usable without predictions. diff --git a/src-tauri/src/prediction/mod.rs b/src-tauri/src/prediction/mod.rs index e4e118e8..a509a5f9 100644 --- a/src-tauri/src/prediction/mod.rs +++ b/src-tauri/src/prediction/mod.rs @@ -184,7 +184,8 @@ struct Service { /// last. It holds no text, and says if a word is in progress when the /// context is lost. shape: Vec, - /// Backspace went past the start of `shape`, into text never seen. + /// Whether the character under the start of `shape` belongs to a word. + /// Text never seen is taken to. beyond: bool, /// The context was lost inside a word, so its remaining letters would be /// completed as if they began one. Nothing reaches the worker until a @@ -261,9 +262,10 @@ impl Service { match edit { Edit::Append(text) => { self.shape.extend(text.chars().map(word_char)); - self.beyond = false; let extra = self.shape.len().saturating_sub(512); - self.shape.drain(..extra); + if let Some(last) = self.shape.drain(..extra).next_back() { + self.beyond = last; + } } Edit::Backspace => { if self.shape.pop().is_none() { @@ -280,9 +282,12 @@ impl Service { } /// Input from outside Switchify may have moved the caret, so what was /// typed before says nothing about where the next letter lands. An - /// observer that is not running cannot tell, and changes nothing. + /// observer that is not running cannot tell, and changes nothing; its + /// own restart is not outside input either. fn observe(&mut self, (epoch, healthy): (u64, bool)) { - if healthy && self.epoch.replace(epoch).is_some_and(|seen| seen != epoch) { + if !healthy { + self.epoch = None; + } else if self.epoch.replace(epoch).is_some_and(|seen| seen != epoch) { self.shape.clear(); self.beyond = false; self.held = false; @@ -326,7 +331,7 @@ impl Service { return; } match &edit { - Edit::Append(text) => self.known += text.chars().count(), + Edit::Append(text) => self.known = (self.known + text.chars().count()).min(512), Edit::Backspace => self.known -= 1, Edit::Reset => {} } @@ -372,8 +377,7 @@ impl Service { fn queue_edit(&mut self, edit: Edit, scope: InputScope) { self.edit_revision = self.edit_revision.wrapping_add(1); if self.edit.len() >= 512 { - self.edit.clear(); - self.reset = true; + self.forget(); } self.edit.push(scope.record(edit)); } @@ -1106,6 +1110,15 @@ mod tests { let mut s = Service::default(); s.stroke(named("Backspace"), true, true, scope); assert!(s.held); + // A space typed and deleted again leaves the caret where it was. + for start in ["End", "Backspace"] { + let mut s = service(""); + s.stroke(named(start), true, true, scope); + s.stroke(key(' '), true, true, scope); + assert!(!s.held); + s.stroke(named("Backspace"), true, true, scope); + assert!(s.held, "{start}"); + } // Deleting what the worker has is an ordinary edit. let mut s = service("hello "); s.stroke(named("Backspace"), true, true, scope); @@ -1175,10 +1188,17 @@ mod tests { let mut s = service("hel"); s.observe((4, true)); s.interrupt(); - // An observer that is not running cannot say anything happened. + s.observe((5, true)); + assert!(!s.held); + // An observer that is not running cannot say anything happened, + // and the epoch it starts again at is its own doing. + let mut s = service("hel"); + s.observe((4, true)); + s.interrupt(); s.observe((5, false)); + s.observe((6, true)); assert!(s.held); - s.observe((5, true)); + s.observe((7, true)); assert!(!s.held); } #[test] @@ -1190,9 +1210,7 @@ mod tests { record_punctuation('.', false, false, InputScope::capture()); SERVICE.with(|slot| { let s = slot.borrow(); - // Outside input during the test would end the hold, not start one. - assert!(s.held || s.shape.is_empty()); - assert!(s.edit.is_empty() && s.reset); + assert!(s.held && s.edit.is_empty() && s.reset); }); stop(); } @@ -1220,6 +1238,11 @@ mod tests { assert_eq!(s.shape, [true, true, true, true, false, true, true, true]); s.typed(Edit::Append("a ".repeat(400)), InputScope::capture()); assert_eq!(s.shape.len(), 512); + assert_eq!(s.known, 512); + // What slid off the start is remembered as the text underneath. + assert!(!s.beyond); + s.typed(Edit::Append("b".into()), InputScope::capture()); + assert!(s.beyond); } #[test] fn invalid_edit_acknowledgements_fail_closed() {