Skip to content

feat(mac-receiver): keyboard and trackpad control of the sending Mac - #25

Closed
raiseCatError wants to merge 4 commits into
feature/per-device-display-modefrom
feature/mac-receiver-input
Closed

raiseCatError wants to merge 4 commits into
feature/per-device-display-modefrom
feature/mac-receiver-input

Conversation

@raiseCatError

Copy link
Copy Markdown
Owner

Refs #12. Stacked on #24: merge #24 first, then retarget this PR. Its diff against feature/per-device-display-mode is only this feature's 3 commits.

What changed

Once the sending Mac grants the session control, the Mac Receiver's own trackpad, mouse and keyboard drive it while the video window is in front, in Mirror and in Extend. It uses the existing pointer, scroll, keyboard and nativeAppGesture messages and the existing per-session request/grant. No new wire message.

Receiver (MacReceiver/ReceiverInputForwarding.swift is pure and tested; ReceiverInputController.swift holds the AppKit glue):

  • Pointer: move, left/right click with click counts, drags clamped to the picture and released even outside the window.
  • Scroll and gestures: precise and wheel scroll in this Mac's scroll direction, and pinch/rotate.
  • Keys: every key of a full keyboard with Shift/Control/Option/Command/Caps Lock, and key repeat.
  • Focus: forwarding needs the grant, the window in front, and no local pause. Input is never captured globally: an app-local monitor scoped to the video window means system shortcuts (Command-Tab, Force Quit, Mission Control, Spotlight) stay local. A modifier held from before forwarding (Command-Tab's Command) is never sent.
  • Release: everything sent down is released on focus loss, the ⌃⌥⎋ pause, Release, revocation and window close. A disconnect resets locally.
  • UI: a titlebar indicator (View Only / Requesting / Controlling / Paused, plus Request Control or Release), a short hint naming the pause shortcut, and the streamed cursor following the local pointer while controlling.

Sender (additive):

  • HID usages for grave, F1–F20, Home/End/Page Up/Down, Help/Insert, keypad, ISO and JIS keys.
  • keyboard down + repeat: true posts an auto-repeat only for a held key.
  • Scroll carries its rounding remainder, so slow trackpad scrolls no longer round away. This also applies to iPhone/iPad.

Docs: RECEIVER_INPUT.md covers behavior, local vs forwarded, guarantees and a 12-step physical checklist. PROTOCOL.md documents the repeat field and the full usage range. README/SETUP/ARCHITECTURE no longer call the Mac Receiver display-only.

Compatibility

  • New receiver + 1.20.0 sender: control works; the new keys and key repeat are ignored by that sender (unknown usages and duplicate downs were always ignored).
  • iPhone/iPad: unchanged. The sendKeyboardDown(usage:modifiers:) signature is preserved, and messages are byte-identical unless repeat is set.

Verification

  • xcodebuild test -scheme OpenSidecarMac (unsigned, Xcode 27.1): 1946 tests, 0 failures (1892 on feat(mac): independent Mirror/Extend per connected device #24's head; 54 new in ReceiverInputForwardingTests).

  • OpenSidecarMacReceiver build ✅. OpenSidecariOS generic-device build ✅. The iOS build caught a protocol-conformance break, which is fixed.

  • The forwarder's decision cost is measured at about 0.45–0.7 µs per pointer event (10,000 moves in 4.2–6.7 ms after warm-up); no threshold is asserted.

  • No test posts events into the live system.

  • Not verified on hardware: real trackpad/keyboard feel, latency, and injected events in real apps. See the checklist in RECEIVER_INPUT.md.

  • The change is focused, without unrelated refactors

  • Relevant behaviour was verified (tests and/or on device): tests only, devices pending

  • Documentation updated if needed

  • No credentials, secrets or signing material included

  • No unrelated generated or build artifacts

…m receivers

A receiving Mac's physical keyboard names keys the sender could not inject:
add HID usages for the grave key, F1-F20, Help/Insert, Home, End, Page
Up/Down, the ISO and JIS keys and the keypad. Older senders ignore them, as
an unrecognized usage must be.

A keyboard down may now carry repeat: true for the receiver's own key
repeat; the sender posts an auto-repeat key-down only while that key is
held. An older sender ignores the duplicate down, as before.

Scroll deltas carry their rounding remainder into the next event, so slow
trackpad scrolls (fractions of a point per event) no longer round away.
A single delta still rounds to nearest, as before.

Refs #12
…sending Mac

The Mac Receiver was display-only. Once the sending Mac grants the session
control through the existing per-session request flow, the video window
forwards pointer movement, left and right clicks with their click counts,
drags (clamped to the picture, released even outside it), precise and
wheel scrolling, pinch and rotate, and every key with Shift, Control,
Option, Command and Caps Lock, using the existing pointer, scroll,
keyboard and nativeAppGesture messages.

Forwarding needs the grant, the video window in front and no local pause.
Input is never captured globally: only events addressed to this app's
video window are looked at, so Command-Tab, Force Quit, Mission Control and
other system shortcuts stay on this Mac. A modifier still held from before
(the Command of a Command-Tab) is never sent. Everything sent down is
released on focus loss, the Control-Option-Escape pause, Release, revocation
or window close; a disconnect resets locally.

A titlebar indicator shows View Only / Requesting / Controlling / Paused
with Request Control or Release, a short hint names the pause shortcut,
and the streamed cursor follows the local pointer while controlling.

Refs #12
Document what is forwarded and what stays local, the pause shortcut, the
release guarantees and a physical test checklist; the additive keyboard
repeat field and the full key range in PROTOCOL.md; and drop the
display-only wording from README, SETUP and ARCHITECTURE.

Refs #12

Copy link
Copy Markdown
Owner Author

Closing as included: this branch's head is an ancestor of dev (merged via #34, merge commit d2e765f), so every commit here is already in the 2.0 development line. The branch itself is left in place.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant