Add point scanning in Settings with reusable native scan logic

docs/point-scan.md

@@ -0,0 +1,27 @@
+# Native point scan
+
+Point scan ports the Android line-only and grid-then-line techniques to Switchify PC. The reference source is `switchifyapp/switchify-android` commit `856720d8747e2f3d1724a572bf754ffae05df299`, especially `PointScanLineManager`, `PointScanBlockManager`, and `ContinuousLineSpeedUtils`.
+
+Open **Settings → Scanning**, choose automatic or manual scanning, and enable it. Local keyboard-emulating switch interfaces can use Space to select, Enter to step forward, Backspace to step backward, and F8 to pause or resume. The four keys can be changed. Escape always disables scanning and releases the reserved keys. Scanning is off at every application startup.
+
+Focus the intended application and press Select to start. Line mode chooses X, then Y, and clicks once. Grid mode chooses a row, then a cell, before the same line sequence. Selecting happens on switch release; holding Select freezes the position and repeat keydowns do not select again. The scan resets after a click and waits for the next Select. Movement wraps at the selected region's edges. Android's five speeds are 45, 75, 120, 180, and 270 logical units per second, with delayed ticks capped at 250 ms.
+
+The scan uses the monitor under the pointer when it starts. Windows uses native physical coordinates and display scaling; macOS uses Core Graphics display units and converts overlay rectangles to AppKit coordinates. A monitor geometry change cancels scanning. Native overlay strips are topmost, click-through, and nonactivating. The pointer moves only for the final click.
+
+This first port uses switches attached to the computer. Android must be disconnected before enabling it, and an Android connection cancels local scanning. Existing authenticated Bluetooth commands and switch-forwarding profiles are unchanged. Input permission is still required on macOS. Point settings use a separate `point-scan.json` in the existing Tauri application configuration directory, so old settings and pairing schemas remain unchanged.
+
+Changes save automatically, including when you leave Settings. Enabling waits for pending saves; a failed save keeps your edits and offers Retry save. Configuration is locked while scanning is enabled. Navigating away does not stop scanning.
+
+The engine is independent of OS input. Activation uses the existing `InputInjector` adapter. Automated tests use a fake adapter and never move the real pointer. The Tauri global-shortcut plugin supplies global press/release events on both platforms; the existing dependencies did not include a switch-key listener. Only the main window can configure scanning through IPC.
+
+Manual validation should cover both modes, every speed, manual movement, holding and releasing switches, Escape, focus retention, display changes, mixed scaling, a Bluetooth connection during scanning, and application exit. Run macOS input checks through `npm run macos:run` to retain the stable Accessibility identity. No physical switch or real desktop input is exercised by automated tests.
+
+## Reusing scanning in Switchify PC
+
+`scanning.rs` is the pure shared core. `Session<T>` owns start, pause, completion, cancellation and bounded automatic timing. `SwitchInput` turns presses and releases into semantic actions, suppresses repeat events and rejects stale shortcut generations. `Cycle` and `Interval` provide traversal and timing. A technique implements `start`, `advance`, `handle`, `reset`, `frame` and `phase`, with its own typed selection result. The test-only item technique exercises this contract without pointer coordinates or desktop input.
+
+`point_scan.rs` implements row/cell and X/Y selection. Its engine receives only point settings; the existing flat `Config` maps into shared switch settings and point settings. `scanning_runtime.rs` owns shortcut registration, session ticking, persistence, event publication and shutdown. Its `Adapter` supplies configuration, environment validation and selection activation. `scan_host.rs` renders shared frame strips through nonactivating Windows and macOS windows. `point_scan_runtime.rs` supplies the display and click adapters, and `point_scan_activation.rs` checks input state before clicking through `InputInjector`.
+
+Only one local technique is installed at a time. A future technique can use the shared controller with its own adapter; adding a technique chooser is separate work. Transport cleanup calls the shared scan service, which invalidates callbacks immediately and releases native resources on the main thread. Queued cancellation cannot stop a newer registration.
+
+The existing `point-scan.json`, `get_point_scan`, `configure_point_scan` and `point-scan-changed` retain their field names and payload shapes. No production item scanner, UI Automation scanner or keyboard integration is added here.

src-tauri/Cargo.toml

@@ -26,6 +26,7 @@ serde_json = "1"
 sha2 = "0.10"
 tauri = { version = "2.11.5", features = ["tray-icon", "image-png", "macos-private-api"] }
 tauri-plugin-autostart = "2.5.1"
+tauri-plugin-global-shortcut = "2"
 tauri-plugin-single-instance = "2.4.3"
 tauri-plugin-updater = "2.10.1"
 tiny-skia = "0.11.4"

src-tauri/build.rs

@@ -5,6 +5,8 @@ fn main() {
     add_command_line_tools_swift_library_path();
     let app_manifest = tauri_build::AppManifest::new().commands(&[
         "get_app_state",
+        "get_point_scan",
+        "configure_point_scan",
         "check_accessibility",
         "approve_pairing",
         "reject_pairing",

src-tauri/capabilities/main.json

@@ -6,6 +6,8 @@
   "permissions": [
     "core:default",
     "allow-get-app-state",
+    "allow-get-point-scan",
+    "allow-configure-point-scan",
     "allow-check-accessibility",
     "allow-approve-pairing",
     "allow-reject-pairing",

src-tauri/permissions/autogenerated/configure_point_scan.toml

@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-configure-point-scan"
+description = "Enables the configure_point_scan command without any pre-configured scope."
+commands.allow = ["configure_point_scan"]
+
+[[permission]]
+identifier = "deny-configure-point-scan"
+description = "Denies the configure_point_scan command without any pre-configured scope."
+commands.deny = ["configure_point_scan"]

src-tauri/permissions/autogenerated/get_point_scan.toml

@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-get-point-scan"
+description = "Enables the get_point_scan command without any pre-configured scope."
+commands.allow = ["get_point_scan"]
+
+[[permission]]
+identifier = "deny-get-point-scan"
+description = "Denies the get_point_scan command without any pre-configured scope."
+commands.deny = ["get_point_scan"]

src-tauri/src/input.rs

@@ -1529,6 +1529,32 @@ mod tests {
         assert_eq!(input.injector.text, vec!["Hello"]);
     }
 
+    #[test]
+    fn point_scan_clicks_once_at_absolute_point_with_fake_input() {
+        let mut input = DesktopInput::new(FakeInjector::default());
+        crate::point_scan_activation::click(&mut input, (-640, 512), true).unwrap();
+        assert_eq!(input.injector.absolute_moves, vec![(-640, 512)]);
+        assert_eq!(input.injector.clicks, vec![(MouseButton::Left, 1)]);
+        assert!(input.injector.keys.is_empty());
+    }
+
+    #[test]
+    fn point_scan_refuses_to_move_during_drag() {
+        let mut input = DesktopInput::new(FakeInjector::default());
+        input.held_button = Some(MouseButton::Left);
+        assert!(crate::point_scan_activation::click(&mut input, (30, 40), true).is_err());
+        assert!(input.injector.absolute_moves.is_empty());
+        assert!(input.injector.clicks.is_empty());
+    }
+
+    #[test]
+    fn point_scan_does_not_inject_a_modified_click() {
+        let mut input = DesktopInput::new(FakeInjector::default());
+        assert!(crate::point_scan_activation::click(&mut input, (30, 40), false).is_err());
+        assert!(input.injector.absolute_moves.is_empty());
+        assert!(input.injector.clicks.is_empty());
+    }
+
     #[cfg(target_os = "windows")]
     #[test]
     fn canonical_windows_alphanumeric_keys_use_physical_keys() {

src-tauri/src/lib.rs

@@ -14,7 +14,13 @@ mod macos_relaunch;
 mod modifier_overlay;
 mod mouse_repeat;
 mod overlay;
+mod point_scan;
+mod point_scan_activation;
+mod point_scan_runtime;
 mod protocol;
+mod scan_host;
+mod scanning;
+mod scanning_runtime;
 mod state;
 mod storage;
 mod telemetry;
@@ -199,6 +205,7 @@ pub(crate) fn sync_tray_state(app: &AppHandle, state: &AppState) {
 }
 
 fn finish_app_exit(app: &AppHandle) {
+    scanning_runtime::cancel(app);
     app.state::<dwell::DwellController>().cancel(app);
     let model = app.state::<AppModel>();
     let _ = platform_disconnect_all(app, &model.shared);
@@ -365,6 +372,7 @@ fn finish_disconnect(
     overlay: &overlay::CursorOverlay,
     modifier_overlay: &modifier_overlay::ModifierOverlay,
 ) -> AppState {
+    scanning_runtime::cancel(app);
     app.state::<dwell::DwellController>().cancel(app);
     overlay.end_session();
     modifier_overlay.end_session();
@@ -1211,6 +1219,54 @@ fn install_tray(app: &mut tauri::App) -> tauri::Result<()> {
     Ok(())
 }
 
+#[tauri::command]
+fn get_point_scan(
+    controller: State<'_, point_scan_runtime::Controller>,
+) -> point_scan_runtime::View {
+    controller.view()
+}
+
+#[tauri::command]
+async fn configure_point_scan(
+    app: AppHandle,
+    config: point_scan::Config,
+    enabled: bool,
+) -> Result<point_scan_runtime::View, String> {
+    let (tx, rx) = tokio::sync::oneshot::channel();
+    let handle = app.clone();
+    app.run_on_main_thread(move || {
+        let _ = tx.send(point_scan_runtime::configure(&handle, config, enabled));
+    })
+    .map_err(|e| e.to_string())?;
+    rx.await
+        .map_err(|_| "Point scan configuration was cancelled.".to_string())?
+}
+
+fn point_scan_prepare(app: &AppHandle) -> Result<(), String> {
+    let state = app.state::<AppModel>().snapshot();
+    if state.bluetooth == state::BluetoothState::Connected {
+        return Err("Disconnect Android before using local point scan.".into());
+    }
+    if state.accessibility != state::AccessibilityState::Granted {
+        return Err("Grant input access before using point scan.".into());
+    }
+    app.state::<dwell::DwellController>().cancel(app);
+    platform_stop_mouse_repeat(app);
+    Ok(())
+}
+
+fn point_scan_click(app: &AppHandle, point: (i32, i32)) -> Result<(), String> {
+    point_scan_prepare(app)?;
+    // Reuse the production input adapter, independent of Bluetooth availability.
+    let injector = enigo::Enigo::new(&enigo::Settings::default())
+        .map_err(|_| "Point scan input could not be initialized.".to_string())?;
+    point_scan_activation::click(
+        &mut input::DesktopInput::new(injector),
+        point,
+        scan_host::modifiers_released(),
+    )
+}
+
 pub fn run() {
     #[cfg(target_os = "macos")]
     if macos_relaunch::run_from_args() {
@@ -1230,6 +1286,7 @@ pub fn run() {
     let overlay_shared = shared.clone();
     let modifier_overlay_shared = shared.clone();
     tauri::Builder::default()
+        .plugin(tauri_plugin_global_shortcut::Builder::new().build())
         .plugin(tauri_plugin_single_instance::init(|app, args, _| {
             if has_start_hidden_argument(&args) {
                 return;
@@ -1250,6 +1307,7 @@ pub fn run() {
         .manage(PendingProfileExit::default())
         .manage(PendingNavigation::default())
         .setup(move |app| {
+            point_scan_runtime::install(app.handle());
             install_tray(app)?;
             if updater_is_configured(app.config().plugins.0.get("updater")) {
                 let model = app.state::<AppModel>();
@@ -1333,6 +1391,8 @@ pub fn run() {
             }
         })
         .invoke_handler(tauri::generate_handler![
+            get_point_scan,
+            configure_point_scan,
             get_app_state,
             check_accessibility,
             approve_pairing,

src-tauri/src/macos.rs

@@ -391,6 +391,7 @@ pub fn reject_pairing(
 }
 
 pub fn disconnect_all(app: &AppHandle, shared: &SharedModel) -> Result<(), String> {
+    crate::scanning_runtime::cancel(app);
     app.state::<DwellController>().cancel(app);
     with_runtime(|runtime| {
         runtime.stop_all_repeats();
@@ -969,6 +970,7 @@ impl MacRuntime {
                     let _ = input.release_all();
                     input.end_control_session();
                 }
+                crate::scanning_runtime::cancel(&self.app);
                 self.app.state::<CursorOverlay>().end_session();
                 self.app.state::<ModifierOverlay>().end_session();
                 self.set_bluetooth(BluetoothState::Advertising);
@@ -1403,6 +1405,7 @@ impl MacRuntime {
                     }
                 }
                 if command.command_type == "connection.disconnecting" {
+                    crate::scanning_runtime::cancel(&self.app);
                     overlay.end_session();
                 }
             }
@@ -2013,6 +2016,7 @@ impl MacRuntime {
             let _ = input.release_all();
             input.end_control_session();
         }
+        crate::scanning_runtime::cancel(&self.app);
         self.app.state::<CursorOverlay>().end_session();
         self.app.state::<ModifierOverlay>().end_session();
     }

src-tauri/src/overlay.rs

@@ -10,10 +10,10 @@ use crate::state::{AppSettings, SharedModel};
 
 #[cfg(target_os = "macos")]
 #[path = "overlay_macos.rs"]
-mod platform;
+pub(crate) mod platform;
 #[cfg(target_os = "windows")]
 #[path = "overlay_windows.rs"]
-mod platform;
+pub(crate) mod platform;
 
 const FOLLOW_INTERVAL: Duration = Duration::from_millis(75);
 const DEFAULT_DURATION: Duration = Duration::from_millis(900);

src-tauri/src/overlay_macos.rs

@@ -190,7 +190,7 @@ impl MacOverlayHost {
     }
 }
 
-fn make_panel(mtm: MainThreadMarker) -> Retained<NSPanel> {
+pub(crate) fn make_panel(mtm: MainThreadMarker) -> Retained<NSPanel> {
     let panel = NSPanel::initWithContentRect_styleMask_backing_defer(
         NSPanel::alloc(mtm),
         rect(0.0, 0.0, 1.0, 1.0),

src-tauri/src/overlay_windows.rs

@@ -206,7 +206,7 @@ unsafe fn create_overlay_window(
     }
 }
 
-fn present_solid(
+pub(crate) fn present_solid(
     window: HWND,
     x: i32,
     y: i32,