Repository navigation
feat(webrtc): accept keyboard and mouse on an input DataChannel - #5
Merged
Merged
Conversation
Browser clients send keyboard, relative and absolute mouse motion, mouse button and scroll events as Moonlight input packets on a new ordered, reliable "input" DataChannel. Only those packet types, at their exact size, are passed to Sunshine's input code, and only while the stream runs, so a client cannot inject controller packets that bypass the gamepad bridge. Clients that do not know the channel close it, which leaves gamepad input untouched. The end-to-end test sends F15 and a pointer move it undoes on the channel, plus a controller packet that must be dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
WebRTC clients could only send gamepad input. This adds keyboard and mouse:
input(ordered, reliable). Clients send each keyboard, mouse motion(relative or absolute), mouse button and scroll event on it as the same binary Moonlight input packet a native
Moonlight client sends.
is_keyboard_mouse_packet()accepts only those packet types at their exact size, with a header that agrees, so aclient cannot inject controller packets that bypass the gamepad bridge and its slots. Accepted packets go to
input::passthrough(), so they behave exactly as with Moonlight (key mapping, modifiers,keyboard/mousesettings, release on disconnect).
The browser client using it is mlopezsegura/moonlight-webrtc-web (
web/input.mjs).Screenshot
Issues Fixed or Closed
Roadmap Issues
Testing
AcceptsOnlyKeyboardAndMousePackets(every accepted type, controller packets, wrong size, truncated,header mismatch) and
ForwardsKeyboardAndMousePacketsWhileTheStreamIsActive.test_sunshine --gtest_filter=Webrtc*:55 passed (Windows, ucrt64 Release build).
tests/webrtc_e2e: the stand-in client now sends F15 down/up, a pointer move it undoes and a controller packet onthe channel. Against an isolated test instance the debug log shows both key packets and both moves reaching
libvirtualhid; the controller packet is dropped.
Ctrl+Alt+Shift+Z release): the server log shows each key, exact motion deltas, X1 button down/up, scroll ±120 and
the release of held modifiers.
Type of Change
Checklist
AI Usage
See our AI usage policy.
🤖 Generated with Claude Code