Skip to content

Windows: real precision-touchpad pinch via Direct Manipulation instead of the Ctrl+wheel emulation #706

Description

@kdroidFilter

Summary

On Windows the Tao backend has no real pinch gesture. A precision touchpad pinch reaches us as the OS's legacy emulation — a Ctrl-flagged WM_MOUSEWHEEL — and the host reassembles a continuous gesture out of that discrete stream by hand. This issue proposes replacing that with Direct Manipulation, which is what Edge / Chromium use for precision-touchpad pan and pinch-zoom on Windows.

Follow-up to #660 / #662, which fixed the dispatch side (a recognized pinch is now ScaleStart / ScaleChange / ScaleEnd at the cursor instead of two synthetic Touch contacts). That change is correct and native on macOS (magnifyWithEvent:) and Linux (GdkEventTouchpadPinch) — both give real begin/update/end phases. Windows is the platform where the input itself is still an emulation.

What we have today

Windows Precision Touchpad converts a pinch into Ctrl+wheel for desktop apps that do not opt into a modern input stack. The vendored Tao MAGNIFY_HOOK patch routes those Ctrl-flagged WM_MOUSEWHEEL messages to the magnify hook instead of the scroll path (decorated-window-tao/src/main/native/src/event_loop.rs, on_tao_magnify), and TaoComposeSceneHostWindows turns the tick stream into a scale gesture.

Everything the host has to invent there comes from the fact that the stream is not a gesture:

  • No phases. There is no Begin and no End, so ScaleStart is synthesised on the first tick and ScaleEnd on a 120 ms idle timer (PINCH_IDLE_END_MS). A burst interrupted by focus loss or a window teardown leaves the gesture open — only the timer closes it.
  • No ratio. TaoWheelPinchZoom.WHEEL_DELTAS_PER_DOUBLING = 12f is a hand-tuned curve mapping notches to a zoom factor. It is not what the fingers did.
  • No focal point. x/y are 0 on the wire; the host centres the gesture on the last cursor position. A real pinch has a centroid that moves independently of the cursor.
  • No inertia, and no way to tell a real mouse's Ctrl+wheel apart from a touchpad pinch — they are the same message.

What Chromium does

Chromium binds a Direct Manipulation viewport to the HWND (IDirectManipulationManager, IDirectManipulationViewport, IDirectManipulationUpdateManager) and drives it from the frame loop. The viewport reports a real transform with begin / update / inertia / end states, so pan and pinch-zoom from a precision touchpad behave like they do on macOS and Linux. Ctrl+wheel from an actual mouse then stays what it is: a discrete zoom command.

Proposed work

  1. A nucleus_tao_windows_dm native piece: create the DM manager, bind a viewport per Tao HWND, configure it for scaling (and decide about panning — see risks), tick the update manager from the existing frame pump.
  2. Bridge the viewport's transform to the wire we already have: TRACKPAD_GESTURE_MAGNIFY with real TRACKPAD_PHASE_BEGAN / CHANGED / ENDED and a real focal point in x/y, so TaoComposeSceneHostWindows can drop schedulePinchEnd / PINCH_IDLE_END_MS and feed TaoTrackpadScaleSession directly — the same shape the macOS and Linux hosts already use.
  3. Keep a fallback: DM is unavailable or refused → the current Ctrl+wheel path, unchanged.
  4. Once a real pinch exists, Ctrl+wheel from a mouse should stop being modelled as a gesture at all (discrete zoom command — tracked separately, it applies to all three platforms).

Risks / open questions

  • DM wants panning too. A viewport configured for interaction can claim the pan stream, which would land on the scroll pipeline we just stabilised in Tao: distinguish trackpad panning from wheel scrolling in Compose #654 / TaoSceneScrollRouter. Configuring scaling-only, or reconciling the two, is the main design question and should be settled before any code.
  • Interaction with the existing WM_POINTER touch path and with the native-view overlay HWNDs (airspace).
  • DM is COM and needs the object lifetime tied to the window's, on the event-loop thread.
  • Needs measuring against the current path: if the OS emulation already delivers enough ticks that a pinch feels smooth, the win is phases + focal point + inertia, not smoothness. Worth a prototype before committing.

Acceptance

  • A precision-touchpad pinch on Windows produces ScaleStart / ScaleChange / ScaleEnd driven by the platform's own phases, with no idle timer anywhere in the path.
  • The focal point follows the fingers, not the cursor.
  • Modifier.transformable and MapLibre zoom identically to macOS.
  • The existing headful coverage still passes (-Dnucleus.tao.headful.filter="#660"), plus a new case driving a real pinch where the runner can inject one.

Out of scope

  • Rotation (Compose has no rotation event; the two-Touch synthesis stays until there is one or until Nucleus exposes its own onRotate).
  • Turning Ctrl+wheel and smart-magnify into discrete zoom commands — separate issue, all platforms.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions