diff --git a/README.md b/README.md index 688e79c07c..52e9fd379f 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,19 @@ # MoonlightVibe -**MoonlightVibe** is the companion Moonlight client for [MultiSeat](https://github.com/vibesoftwarecoder/MultiSeat) — a system that runs multiple simultaneous game streaming sessions on one Windows host. +**MoonlightVibe** is a Moonlight game streaming client. It is the companion client for [MultiSeat](https://github.com/vibesoftwarecoder/MultiSeat), which runs several streaming sessions on one Windows host, and it also works with any Moonlight-compatible host. -It is a fork of [moonlight-stream/moonlight-qt](https://github.com/moonlight-stream/moonlight-qt) with two additions: MultiSeat seat auto-discovery and microphone passthrough support. +It is a fork of [moonlight-stream/moonlight-qt](https://github.com/moonlight-stream/moonlight-qt). This fork adds seat auto-discovery, microphone passthrough, USB forwarding and a second window. All four need [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) on the host, except seat auto-discovery, which needs MultiSeat. --- ## Where to report a problem -**Anything about the client goes here** — including problems inherited from upstream Moonlight. +**Anything about the client goes here**, including problems inherited from upstream Moonlight. MoonlightVibe is what you installed, so it is ours to answer for. | what is wrong | where it goes | |---|---| -| The client: streaming, decoding, seat discovery, gamepad, microphone passthrough | **here** | +| The client: streaming, decoding, seat discovery, gamepad, microphone, USB forwarding, the second window | **here** | | Picture, encoder or host audio | [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe/issues) | | A seat will not provision | [MultiSeat](https://github.com/vibesoftwarecoder/MultiSeat/issues) | @@ -24,11 +24,13 @@ not maintain this fork, and cannot help with it. | Feature | MoonlightVibe | Upstream Moonlight | |---|---|---| -| MultiSeat auto-discovery | ✅ Seats appear automatically in the computer list | ❌ Manual host entry required | -| Mic passthrough | ✅ Client mic streamed to host via ApolloVibe | ❌ Not available | +| Seat auto-discovery | ✅ MultiSeat seats appear in the computer list by themselves | ❌ Add each host by hand | +| Microphone passthrough | ✅ Your microphone is streamed to the host | ❌ Not available | +| Second window | ✅ A second picture from the same session, in a window of its own: one host window, or a whole display | ❌ Not available | +| USB forwarding | ✅ Share a USB device with the host (Windows, experimental, off by default) | ❌ Not available | | Branding | MoonlightVibe | Moonlight | -Everything else — hardware video decoding, H.264/HEVC/AV1, HDR, surround sound, gamepad support, touch input — is inherited from upstream Moonlight. +Everything else is inherited from upstream Moonlight: hardware video decoding, H.264, HEVC and AV1, HDR, surround sound, gamepads and touch input. --- @@ -36,72 +38,116 @@ Everything else — hardware video decoding, H.264/HEVC/AV1, HDR, surround sound Get the latest release from the [Releases page](https://github.com/vibesoftwarecoder/MoonlightVibe/releases/latest). -Extract the zip and run `MoonlightVibe.exe` — no installer required, fully portable. +Extract the zip and run `MoonlightVibe.exe`. There is no installer, and it is fully portable. --- -## Code signing policy +## Second window -Free code signing provided by [SignPath.io](https://about.signpath.io), certificate by -[SignPath Foundation](https://signpath.org). +A session can show a second picture of the host in a second client window. It is the same connection and the same pairing, and nothing extra to run on the host. The second picture is one of two things: -- **What is signed:** the two files built from this repository, `MoonlightVibe.exe` and - `AntiHooking.dll`, in the Windows release. They show "SignPath Foundation" as the publisher. -- **What is not:** third-party libraries bundled from other open-source projects, such as FFmpeg, - SDL and OpenSSL, are included as those projects or their packagers provide them. Qt's libraries - carry The Qt Company's own signature. -- **How:** every Windows release is built from this repository by GitHub Actions on GitHub-hosted - runners, and each signing request is approved by hand before it is signed. +- **A window on the host.** You name the window by its title, for example a browser or a chat. Only that window is shown, and it follows the window if it moves. +- **A whole display.** The host makes an extra virtual monitor. Drag windows onto it in the main window and they show up in the second one, so it works like a real second screen. -Team roles: +**To use it:** open **Settings → Second Window**, turn on **Show a second window**, and choose what to show. Then start a stream as usual. The options are: -- Committers and reviewers: [vibesoftwarecoder](https://github.com/vibesoftwarecoder) -- Approvers: [vibesoftwarecoder](https://github.com/vibesoftwarecoder) +- **What to show:** a window, or a whole display. For a window, type its title. A title that matches exactly is used first, then one that contains your text. +- **Resolution and frame rate** of the second picture: 720p to 4K, at 30 or 60 frames per second. A bigger picture is sharper when the window is large, and uses more bandwidth. It can't add detail the host window doesn't have. +- **Show it on this display:** which of your screens the second window opens on. Automatic picks a screen the main window is not on. +- **Open the second window in full screen:** on by default. Press **F11** in the second window to switch between full screen and a window. -Privacy: see [PRIVACY.md](PRIVACY.md). MoonlightVibe sends no data to us. It does contact some -servers run by others, such as GitHub for its update check, and that page lists each one. +Closing the second window stops only the second picture. The main window keeps streaming. + +**Command line.** The same options work from the command line: + +``` +MoonlightVibe.exe stream --second-window "Window title" +MoonlightVibe.exe stream --second-window-source display +``` + +Also: `--second-window-resolution WxH`, `--second-window-fps N`, `--second-window-display N`, `--second-window-fullscreen` or `--no-second-window-fullscreen`, and `--window-display N` for the main window. + +**What it needs:** + +- An [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) host with `second_video_stream = enabled`. A host without it ignores the request, and you get one window. +- **A whole display needs the console host**, the main desktop of the machine. A MultiSeat seat can only show a window. + +**Known limits.** The second window follows the mouse by absolute position only, with no relative mouse mode inside it. If the host window is not 16:9, the picture has black bars. --- ## MultiSeat seat auto-discovery -When MultiSeat is running on the local machine, MoonlightVibe automatically discovers all active seats and lists each one as a separate server in the computer list. No manual host entry or port configuration needed — seats appear within ~15 seconds of becoming ready. +When MultiSeat is running on the local machine, MoonlightVibe finds all active seats and lists each one as its own server in the computer list. You don't enter a host or a port. Seats appear within about 15 seconds of becoming ready. -This works over the LAN too: if you point MoonlightVibe at the MultiSeat host IP, it will discover and list all seats on that machine. +This works over the LAN too. Point MoonlightVibe at the MultiSeat host's address, and it lists all the seats on that machine. --- ## Microphone passthrough -MoonlightVibe streams your microphone to the host and plays it back through `Speakers (Steam Streaming Microphone)`. Games and apps on the host should select `Microphone (Steam Streaming Microphone)` as their input device. +MoonlightVibe streams your microphone to the host, which plays it into `Speakers (Steam Streaming Microphone)`. Apps and games on the host should pick `Microphone (Steam Streaming Microphone)` as their input. + +It needs [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) on the host with `stream_mic = enabled`. MultiSeat turns that on for you. + +--- + +## USB forwarding (experimental) + +Windows only, and off by default. Open **Settings → USB forwarding** to share a USB device from your PC so it appears on the host you are streaming to. You can share or stop sharing a device, choose to forward it automatically when streaming, and press **Ctrl+Alt+Shift+U** to pause or resume. With the setting off, nothing is created and a stream behaves exactly as before. -Requires [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) on the host side with `stream_mic = enabled` in the seat config. MultiSeat enables this automatically. +It needs `usbipd-win` on your PC, and an ApolloVibe host with USB forwarding turned on, the `usbip-win2` driver, and the USB permission granted to your client. Sharing a device asks for administrator approval. Try a USB stick or a spare gamepad first, not a keyboard or mouse. + +It builds on the work of qiin2333 and AlkaidLab. See the [6.3.11 release notes](docs/release-notes/6.3.11.md) for details. + +--- ## Sharing diagnostic logs -On Windows and macOS, open **Settings → Advanced Settings** and turn on **Enable detailed logging** before reproducing a problem. Then select **Open Log Folder** and attach the newest `Moonlight-*.log` file to your issue. Warning and error messages are logged even when detailed logging is off. The setting persists until you turn it off. +On Windows and macOS, open **Settings → Advanced Settings** and turn on **Enable detailed logging** before reproducing a problem. Then select **Open Log Folder** and attach the newest `Moonlight-*.log` file to your issue. Warning and error messages are logged even when detailed logging is off. The setting stays on until you turn it off. -Please review a log before sharing it: it may include your device or host names, network addresses, and details of your streaming session. Session encryption keys are removed automatically. +Please read a log before sharing it. It may include your device or host names, network addresses, and details of your streaming session. Session encryption keys are removed automatically. --- -## Requirements +## Code signing policy + +Free code signing provided by [SignPath.io](https://about.signpath.io), certificate by +[SignPath Foundation](https://signpath.org). + +- **What is signed:** the two files built from this repository, `MoonlightVibe.exe` and + `AntiHooking.dll`, in the Windows release. They show "SignPath Foundation" as the publisher. +- **What is not:** third-party libraries bundled from other open-source projects, such as FFmpeg, + SDL and OpenSSL, are included as those projects or their packagers provide them. Qt's libraries + carry The Qt Company's own signature. +- **How:** every Windows release is built from this repository by GitHub Actions on GitHub-hosted + runners, and each signing request is approved by hand before it is signed. -- Windows 10/11 x64 -- [MultiSeat](https://github.com/vibesoftwarecoder/MultiSeat) + [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) on the host for full feature support -- Standard Moonlight-compatible host (e.g. upstream Sunshine) works for basic streaming without mic or auto-discovery +Team roles: + +- Committers and reviewers: [vibesoftwarecoder](https://github.com/vibesoftwarecoder) +- Approvers: [vibesoftwarecoder](https://github.com/vibesoftwarecoder) + +Privacy: see [PRIVACY.md](PRIVACY.md). MoonlightVibe sends no data to us. It does contact some +servers run by others, such as GitHub for its update check, and that page lists each one. --- -## Usage with standard Moonlight hosts +## Requirements + +- Windows 10 or 11, x64. +- For the extra features, [ApolloVibe](https://github.com/vibesoftwarecoder/ApolloVibe) on the host. Seat auto-discovery also needs [MultiSeat](https://github.com/vibesoftwarecoder/MultiSeat). +- A standard Moonlight-compatible host, such as upstream Sunshine or Apollo, works for ordinary streaming. It does not offer the microphone, the second window or USB forwarding. + +## Using standard Moonlight hosts -MoonlightVibe is fully compatible with any Moonlight-compatible streaming host (Sunshine, upstream Apollo, NVIDIA GameStream). Auto-discovery only activates when a MultiSeat service is detected. Mic passthrough requires ApolloVibe with `stream_mic = enabled`. +MoonlightVibe is fully compatible with Sunshine, upstream Apollo and NVIDIA GameStream. Seat auto-discovery only turns on when a MultiSeat service is found. The microphone, the second window and USB forwarding each need an ApolloVibe host with that feature enabled, and are ignored by other hosts. --- ## Building -MoonlightVibe uses the same build system as upstream moonlight-qt — Qt 6.7+ and Visual Studio 2022 on Windows. +MoonlightVibe builds the same way as upstream moonlight-qt: Qt 6.7 or newer and Visual Studio 2022 on Windows. See the upstream [build instructions](https://github.com/moonlight-stream/moonlight-qt#building) for the full dependency list. @@ -111,4 +157,4 @@ See the upstream [build instructions](https://github.com/moonlight-stream/moonli MoonlightVibe is licensed under the same terms as Moonlight — [GPL-3.0](LICENSE). -Mic passthrough patches by [logabell](https://github.com/logabell/moonlight-qt-mic). +Microphone passthrough patches by [logabell](https://github.com/logabell/moonlight-qt-mic). diff --git a/app/app.pro b/app/app.pro index f5c5a0d653..eeadc5e65f 100644 --- a/app/app.pro +++ b/app/app.pro @@ -193,6 +193,7 @@ SOURCES += \ streaming/input/mouse.cpp \ streaming/input/reltouch.cpp \ streaming/session.cpp \ + streaming/secondarystream.cpp \ streaming/audio/audio.cpp \ streaming/audio/capture/microphonecapture.cpp \ streaming/audio/renderers/sdlaud.cpp \ @@ -239,6 +240,7 @@ HEADERS += \ settings/streamingpreferences.h \ streaming/input/input.h \ streaming/session.h \ + streaming/secondarystream.h \ streaming/audio/capture/microphonecapture.h \ streaming/audio/renderers/renderer.h \ streaming/audio/renderers/sdl.h \ diff --git a/app/cli/commandlineparser.cpp b/app/cli/commandlineparser.cpp index 2acac458a5..f01ad10971 100644 --- a/app/cli/commandlineparser.cpp +++ b/app/cli/commandlineparser.cpp @@ -354,7 +354,14 @@ void StreamCommandLineParser::parse(const QStringList &args, StreamingPreference parser.addChoiceOption("audio-config", "audio config", m_AudioConfigMap.keys()); parser.addToggleOption("multi-controller", "multiple controller support"); parser.addToggleOption("quit-after", "quit app after session"); + parser.addToggleOption("second-window-fullscreen", "open the second window in full screen"); parser.addToggleOption("absolute-mouse", "remote desktop optimized mouse control"); + parser.addValueOption("second-window", "title of a host window to show in a second client window"); + parser.addValueOption("window-display", "display number (1, 2, ...) for the stream window, 0 = automatic"); + parser.addValueOption("second-window-source", "what the second window shows: window (a host window, the default) or display (a display the host makes)"); + parser.addValueOption("second-window-resolution", "custom x resolution of the second window's video"); + parser.addValueOption("second-window-fps", "frame rate of the second window's video"); + parser.addValueOption("second-window-display", "display number (1, 2, ...) for the second window, 0 = automatic"); parser.addToggleOption("mouse-buttons-swap", "left and right mouse buttons swap"); parser.addToggleOption("touchscreen-trackpad", "touchscreen in trackpad mode"); parser.addToggleOption("game-optimization", "game optimizations"); @@ -448,6 +455,44 @@ void StreamCommandLineParser::parse(const QStringList &args, StreamingPreference // Resolve --quit-after and --no-quit-after options preferences->quitAppAfter = parser.getToggleOptionValue("quit-after", preferences->quitAppAfter); + preferences->secondWindowFullscreen = parser.getToggleOptionValue("second-window-fullscreen", preferences->secondWindowFullscreen); + + // Resolve --second-window option + if (parser.isSet("second-window")) { + preferences->secondWindowTitle = parser.value("second-window"); + preferences->secondWindowEnabled = true; + } + if (parser.isSet("second-window-source")) { + QString source = parser.value("second-window-source").toLower(); + if (source == "display") { + preferences->secondWindowSource = 1; + preferences->secondWindowEnabled = true; + } + else if (source == "window") { + preferences->secondWindowSource = 0; + } + else { + parser.showError(QString("Invalid second-window-source choice: %1").arg(source)); + } + } + + // Resolve --second-window-resolution and --second-window-fps options + if (parser.isSet("second-window-resolution")) { + auto resolution = parser.getResolutionOptionValue("second-window-resolution"); + preferences->secondWindowWidth = resolution.first; + preferences->secondWindowHeight = resolution.second; + } + if (parser.isSet("second-window-fps")) { + preferences->secondWindowFps = parser.getIntOption("second-window-fps"); + } + + // Resolve --window-display and --second-window-display options + if (parser.isSet("window-display")) { + preferences->primaryWindowDisplay = qMax(0, parser.getIntOption("window-display")); + } + if (parser.isSet("second-window-display")) { + preferences->secondWindowDisplay = qMax(0, parser.getIntOption("second-window-display")); + } // Resolve --absolute-mouse and --no-absolute-mouse options preferences->absoluteMouseMode = parser.getToggleOptionValue("absolute-mouse", preferences->absoluteMouseMode); diff --git a/app/gui/SettingsView.qml b/app/gui/SettingsView.qml index 373a2890b4..32af7e46a8 100644 --- a/app/gui/SettingsView.qml +++ b/app/gui/SettingsView.qml @@ -817,6 +817,81 @@ Flickable { ToolTip.text: qsTr("Fullscreen generally provides the best performance, but borderless windowed may work better with features like macOS Spaces, Alt+Tab, screenshot tools, on-screen overlays, etc.") } + Label { + width: parent.width + id: primaryWindowDisplayTitle + text: qsTr("Stream window display") + font.pointSize: 12 + wrapMode: Text.Wrap + visible: SystemProperties.hasDesktopEnvironment + } + + AutoResizingComboBox { + function createModel() { + var model = Qt.createQmlObject('import QtQuick 2.0; ListModel {}', parent, '') + + model.append({ + text: qsTr("Automatic"), + val: 0 + }) + + // Refresh display data before using it to build the list + SystemProperties.refreshDisplays() + for (var displayIndex = 0; ; displayIndex++) { + var rect = SystemProperties.getNativeResolution(displayIndex) + if (rect.width <= 0 || rect.height <= 0) { + break + } + + model.append({ + text: qsTr("Display %1").arg(displayIndex + 1) + " (" + rect.width + "x" + rect.height + ")", + val: displayIndex + 1 + }) + } + + return model + } + + // This is used on initialization and upon retranslation + function reinitialize() { + if (!visible) { + return + } + + model = createModel() + currentIndex = 0 + + // Set the current value based on the saved preferences + var saved = StreamingPreferences.primaryWindowDisplay + for (var i = 0; i < model.count; i++) { + if (saved === model.get(i).val) { + currentIndex = i + break + } + } + + activated(currentIndex) + } + + Component.onCompleted: { + reinitialize() + languageChanged.connect(reinitialize) + } + + id: primaryWindowDisplayComboBox + visible: SystemProperties.hasDesktopEnvironment + hoverEnabled: true + textRole: "text" + onActivated: { + StreamingPreferences.primaryWindowDisplay = model.get(currentIndex).val + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Which display the stream window opens on. Automatic uses the display this app is on.") + } + Row { spacing: 5 width: parent.width @@ -879,6 +954,345 @@ Flickable { } } + GroupBox { + id: secondWindowGroupBox + width: (parent.width - (parent.leftPadding + parent.rightPadding)) + padding: 12 + title: "" + qsTr("Second Window") + "" + font.pointSize: 12 + visible: SystemProperties.hasDesktopEnvironment + + Column { + anchors.fill: parent + spacing: 5 + + CheckBox { + id: secondWindowCheck + width: parent.width + text: qsTr("Show a second window") + font.pointSize: 12 + checked: StreamingPreferences.secondWindowEnabled + onCheckedChanged: { + StreamingPreferences.secondWindowEnabled = checked + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Streams a second picture of the host, a window, in a window of its own. The host must be running ApolloVibe with the second video stream turned on; otherwise you get one window.") + } + + Column { + id: secondWindowOptions + width: parent.width + spacing: 5 + visible: secondWindowCheck.checked + + Label { + width: parent.width + id: secondWindowSourceLabel + text: qsTr("What to show") + font.pointSize: 12 + wrapMode: Text.Wrap + } + + AutoResizingComboBox { + function createModel() { + var model = Qt.createQmlObject('import QtQuick 2.0; ListModel {}', parent, '') + + model.append({ text: qsTr("A window on the host"), val: 0 }) + model.append({ text: qsTr("A whole display (the host makes it)"), val: 1 }) + + return model + } + + function reinitialize() { + if (!visible) { + return + } + + model = createModel() + currentIndex = 0 + + for (var i = 0; i < model.count; i++) { + if (StreamingPreferences.secondWindowSource === model.get(i).val) { + currentIndex = i + break + } + } + + activated(currentIndex) + } + + Component.onCompleted: { + reinitialize() + languageChanged.connect(reinitialize) + } + + id: secondWindowSourceComboBox + onVisibleChanged: if (visible) reinitialize() + hoverEnabled: true + textRole: "text" + onActivated: { + StreamingPreferences.secondWindowSource = model.get(currentIndex).val + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("A window shows one window of the host. A whole display makes an extra monitor on the host that you can drag apps onto; it needs a host that can make a virtual display, such as a console, not a seat.") + } + + Label { + width: parent.width + text: qsTr("A whole display needs the console host. On a seat, choose a window.") + font.pointSize: 9 + wrapMode: Text.Wrap + opacity: 0.7 + } + + Label { + width: parent.width + id: secondWindowTitleLabel + visible: StreamingPreferences.secondWindowSource === 0 + text: qsTr("Title of the host window") + font.pointSize: 12 + wrapMode: Text.Wrap + } + + TextField { + id: secondWindowTitleField + visible: StreamingPreferences.secondWindowSource === 0 + width: parent.width + hoverEnabled: true + placeholderText: qsTr("Empty means no second window") + text: StreamingPreferences.secondWindowTitle + onTextEdited: { + StreamingPreferences.secondWindowTitle = text.trim() + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Type the title of a window on the host to show it in a second window. Leave it empty for one window. A host that does not offer a second stream ignores this.") + } + + + Label { + width: parent.width + id: secondWindowResolutionTitle + text: qsTr("Resolution") + font.pointSize: 12 + wrapMode: Text.Wrap + } + + AutoResizingComboBox { + function createModel() { + var model = Qt.createQmlObject('import QtQuick 2.0; ListModel {}', parent, '') + + model.append({ text: "1280x720", w: 1280, h: 720 }) + model.append({ text: "1920x1080", w: 1920, h: 1080 }) + model.append({ text: "2560x1440", w: 2560, h: 1440 }) + model.append({ text: "3840x2160", w: 3840, h: 2160 }) + + return model + } + + function reinitialize() { + if (!visible) { + return + } + + model = createModel() + currentIndex = 1 + + for (var i = 0; i < model.count; i++) { + if (StreamingPreferences.secondWindowWidth === model.get(i).w && + StreamingPreferences.secondWindowHeight === model.get(i).h) { + currentIndex = i + break + } + } + + activated(currentIndex) + } + + Component.onCompleted: { + reinitialize() + } + + id: secondWindowResolutionComboBox + onVisibleChanged: if (visible) reinitialize() + hoverEnabled: true + textRole: "text" + onActivated: { + StreamingPreferences.secondWindowWidth = model.get(currentIndex).w + StreamingPreferences.secondWindowHeight = model.get(currentIndex).h + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("The size of the video of the second window. A larger size is sharper when the window is large or fullscreen, and uses more bandwidth.") + } + + + Label { + width: parent.width + id: secondWindowFpsTitle + text: qsTr("Frame rate") + font.pointSize: 12 + wrapMode: Text.Wrap + } + + AutoResizingComboBox { + function createModel() { + var model = Qt.createQmlObject('import QtQuick 2.0; ListModel {}', parent, '') + + model.append({ text: "30 FPS", val: 30 }) + model.append({ text: "60 FPS", val: 60 }) + + return model + } + + function reinitialize() { + if (!visible) { + return + } + + model = createModel() + currentIndex = 1 + + for (var i = 0; i < model.count; i++) { + if (StreamingPreferences.secondWindowFps === model.get(i).val) { + currentIndex = i + break + } + } + + activated(currentIndex) + } + + Component.onCompleted: { + reinitialize() + } + + id: secondWindowFpsComboBox + onVisibleChanged: if (visible) reinitialize() + hoverEnabled: true + textRole: "text" + onActivated: { + StreamingPreferences.secondWindowFps = model.get(currentIndex).val + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("The frame rate of the video of the second window. 30 is enough for most windows and uses less bandwidth.") + } + + + Label { + width: parent.width + id: secondWindowDisplayTitle + text: qsTr("Show it on this display") + font.pointSize: 12 + wrapMode: Text.Wrap + } + + AutoResizingComboBox { + function createModel() { + var model = Qt.createQmlObject('import QtQuick 2.0; ListModel {}', parent, '') + + model.append({ + text: qsTr("Automatic"), + val: 0 + }) + + // Refresh display data before using it to build the list + SystemProperties.refreshDisplays() + for (var displayIndex = 0; ; displayIndex++) { + var rect = SystemProperties.getNativeResolution(displayIndex) + if (rect.width <= 0 || rect.height <= 0) { + break + } + + model.append({ + text: qsTr("Display %1").arg(displayIndex + 1) + " (" + rect.width + "x" + rect.height + ")", + val: displayIndex + 1 + }) + } + + return model + } + + // This is used on initialization and upon retranslation + function reinitialize() { + if (!visible) { + return + } + + model = createModel() + currentIndex = 0 + + // Set the current value based on the saved preferences + var saved = StreamingPreferences.secondWindowDisplay + for (var i = 0; i < model.count; i++) { + if (saved === model.get(i).val) { + currentIndex = i + break + } + } + + activated(currentIndex) + } + + Component.onCompleted: { + reinitialize() + languageChanged.connect(reinitialize) + } + + id: secondWindowDisplayComboBox + onVisibleChanged: if (visible) reinitialize() + hoverEnabled: true + textRole: "text" + onActivated: { + StreamingPreferences.secondWindowDisplay = model.get(currentIndex).val + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Which display the second window opens on, when a second window is used. Automatic picks a display the stream window is not on.") + } + + CheckBox { + id: secondWindowFullscreenCheck + width: parent.width + text: qsTr("Open the second window in full screen") + font.pointSize: 12 + checked: StreamingPreferences.secondWindowFullscreen + onCheckedChanged: { + StreamingPreferences.secondWindowFullscreen = checked + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Opens the second window borderless on the display it goes to, as if F11 had already been pressed. F11 switches it back to a window.") + } + + Label { + width: parent.width + text: qsTr("Press F11 in the second window to switch it to full screen and back.") + font.pointSize: 9 + wrapMode: Text.Wrap + opacity: 0.7 + } + } + } + } + GroupBox { id: audioSettingsGroupBox @@ -1576,6 +1990,24 @@ Flickable { ToolTip.visible: hovered ToolTip.text: qsTr("Prevents the screensaver from starting or the display from going to sleep while streaming.") } + + CheckBox { + id: showPerformanceOverlay + width: parent.width + text: qsTr("Show performance stats while streaming") + font.pointSize: 12 + checked: StreamingPreferences.showPerformanceOverlay + onCheckedChanged: { + StreamingPreferences.showPerformanceOverlay = checked + } + + ToolTip.delay: 1000 + ToolTip.timeout: 5000 + ToolTip.visible: hovered + ToolTip.text: qsTr("Display real-time stream performance information while streaming.") + "\n\n" + + qsTr("You can toggle it at any time while streaming using Ctrl+Alt+Shift+S or Select+L1+R1+X.") + "\n\n" + + qsTr("The performance overlay is not supported on Steam Link or Raspberry Pi.") + } } } } @@ -2103,24 +2535,6 @@ Flickable { } } - CheckBox { - id: showPerformanceOverlay - width: parent.width - text: qsTr("Show performance stats while streaming") - font.pointSize: 12 - checked: StreamingPreferences.showPerformanceOverlay - onCheckedChanged: { - StreamingPreferences.showPerformanceOverlay = checked - } - - ToolTip.delay: 1000 - ToolTip.timeout: 5000 - ToolTip.visible: hovered - ToolTip.text: qsTr("Display real-time stream performance information while streaming.") + "\n\n" + - qsTr("You can toggle it at any time while streaming using Ctrl+Alt+Shift+S or Select+L1+R1+X.") + "\n\n" + - qsTr("The performance overlay is not supported on Steam Link or Raspberry Pi.") - } - CheckBox { id: diagnosticLoggingCheck width: parent.width diff --git a/app/settings/streamingpreferences.cpp b/app/settings/streamingpreferences.cpp index cce7a56f6c..025adea610 100644 --- a/app/settings/streamingpreferences.cpp +++ b/app/settings/streamingpreferences.cpp @@ -39,6 +39,15 @@ #define SER_CONNWARNINGS "connwarnings" #define SER_CONFWARNINGS "confwarnings" #define SER_UIDISPLAYMODE "uidisplaymode" +#define SER_PRIMARYWINDOWDISPLAY "primarywindowdisplay" +#define SER_SECONDWINDOWDISPLAY "secondwindowdisplay" +#define SER_SECONDWINDOWWIDTH "secondwindowwidth" +#define SER_SECONDWINDOWHEIGHT "secondwindowheight" +#define SER_SECONDWINDOWFPS "secondwindowfps" +#define SER_SECONDWINDOWTITLE "secondwindowtitle" +#define SER_SECONDWINDOWENABLED "secondwindowenabled" +#define SER_SECONDWINDOWSOURCE "secondwindowsource" +#define SER_SECONDWINDOWFULLSCREEN "secondwindowfullscreen" #define SER_RICHPRESENCE "richpresence" #define SER_GAMEPADMOUSE "gamepadmouse" #define SER_DEFAULTVER "defaultver" @@ -196,6 +205,15 @@ void StreamingPreferences::reload() uiDisplayMode = static_cast(settings.value(SER_UIDISPLAYMODE, static_cast(settings.value(SER_STARTWINDOWED, true).toBool() ? UIDisplayMode::UI_WINDOWED : UIDisplayMode::UI_MAXIMIZED)).toInt()); + primaryWindowDisplay = settings.value(SER_PRIMARYWINDOWDISPLAY, 0).toInt(); + secondWindowDisplay = settings.value(SER_SECONDWINDOWDISPLAY, 0).toInt(); + secondWindowWidth = settings.value(SER_SECONDWINDOWWIDTH, 1920).toInt(); + secondWindowHeight = settings.value(SER_SECONDWINDOWHEIGHT, 1080).toInt(); + secondWindowFps = settings.value(SER_SECONDWINDOWFPS, 60).toInt(); + secondWindowTitle = settings.value(SER_SECONDWINDOWTITLE, QString()).toString(); + secondWindowEnabled = settings.value(SER_SECONDWINDOWENABLED, false).toBool(); + secondWindowSource = settings.value(SER_SECONDWINDOWSOURCE, 0).toInt(); + secondWindowFullscreen = settings.value(SER_SECONDWINDOWFULLSCREEN, true).toBool(); language = static_cast(settings.value(SER_LANGUAGE, static_cast(Language::LANG_AUTO)).toInt()); @@ -384,6 +402,15 @@ void StreamingPreferences::save() settings.setValue(SER_RENDERER, static_cast(rendererSelection)); settings.setValue(SER_WINDOWMODE, static_cast(windowMode)); settings.setValue(SER_UIDISPLAYMODE, static_cast(uiDisplayMode)); + settings.setValue(SER_PRIMARYWINDOWDISPLAY, primaryWindowDisplay); + settings.setValue(SER_SECONDWINDOWDISPLAY, secondWindowDisplay); + settings.setValue(SER_SECONDWINDOWWIDTH, secondWindowWidth); + settings.setValue(SER_SECONDWINDOWHEIGHT, secondWindowHeight); + settings.setValue(SER_SECONDWINDOWFPS, secondWindowFps); + settings.setValue(SER_SECONDWINDOWTITLE, secondWindowTitle); + settings.setValue(SER_SECONDWINDOWENABLED, secondWindowEnabled); + settings.setValue(SER_SECONDWINDOWSOURCE, secondWindowSource); + settings.setValue(SER_SECONDWINDOWFULLSCREEN, secondWindowFullscreen); settings.setValue(SER_LANGUAGE, static_cast(language)); settings.setValue(SER_DEFAULTVER, CURRENT_DEFAULT_VER); settings.setValue(SER_SWAPMOUSEBUTTONS, swapMouseButtons); diff --git a/app/settings/streamingpreferences.h b/app/settings/streamingpreferences.h index c073dbdf03..ae1d54afad 100644 --- a/app/settings/streamingpreferences.h +++ b/app/settings/streamingpreferences.h @@ -177,6 +177,15 @@ class StreamingPreferences : public QObject Q_PROPERTY(WindowMode windowMode MEMBER windowMode NOTIFY windowModeChanged) Q_PROPERTY(WindowMode recommendedFullScreenMode MEMBER recommendedFullScreenMode CONSTANT) Q_PROPERTY(UIDisplayMode uiDisplayMode MEMBER uiDisplayMode NOTIFY uiDisplayModeChanged) + Q_PROPERTY(int primaryWindowDisplay MEMBER primaryWindowDisplay NOTIFY primaryWindowDisplayChanged) + Q_PROPERTY(int secondWindowDisplay MEMBER secondWindowDisplay NOTIFY secondWindowDisplayChanged) + Q_PROPERTY(int secondWindowWidth MEMBER secondWindowWidth NOTIFY secondWindowVideoChanged) + Q_PROPERTY(int secondWindowHeight MEMBER secondWindowHeight NOTIFY secondWindowVideoChanged) + Q_PROPERTY(int secondWindowFps MEMBER secondWindowFps NOTIFY secondWindowVideoChanged) + Q_PROPERTY(bool secondWindowEnabled MEMBER secondWindowEnabled NOTIFY secondWindowEnabledChanged) + Q_PROPERTY(int secondWindowSource MEMBER secondWindowSource NOTIFY secondWindowSourceChanged) + Q_PROPERTY(bool secondWindowFullscreen MEMBER secondWindowFullscreen NOTIFY secondWindowFullscreenChanged) + Q_PROPERTY(QString secondWindowTitle MEMBER secondWindowTitle NOTIFY secondWindowTitleChanged) Q_PROPERTY(bool swapMouseButtons MEMBER swapMouseButtons NOTIFY mouseButtonsChanged) Q_PROPERTY(bool muteOnFocusLoss MEMBER muteOnFocusLoss NOTIFY muteOnFocusLossChanged) Q_PROPERTY(bool usbForwardingEnabled MEMBER usbForwardingEnabled NOTIFY usbForwardingEnabledChanged) @@ -213,6 +222,17 @@ class StreamingPreferences : public QObject bool enableMdns; bool quitAppAfter; bool absoluteMouseMode; + + // Title of a window on the host to show in a second client window. Saved, and --second-window overrides it for + // a run; a per-seat setting fills it in when empty. Empty means no second window. A host that does not offer a + // second stream is never asked for one, so a saved title is harmless there. + // The Settings check box. --second-window switches it on for a run. + bool secondWindowEnabled; + // What the second window shows: 0 = a window of the host (by title), 1 = a whole display the host makes + int secondWindowSource; + // Open the second window in full screen, as if F11 had been pressed + bool secondWindowFullscreen; + QString secondWindowTitle; bool absoluteTouchMode; bool framePacing; bool connectionWarnings; @@ -242,6 +262,15 @@ class StreamingPreferences : public QObject WindowMode windowMode; WindowMode recommendedFullScreenMode; UIDisplayMode uiDisplayMode; + + // Which display each stream window opens on: 0 picks one automatically, 1 is the first display, 2 the second + int primaryWindowDisplay; + int secondWindowDisplay; + + // The video stream of the second window: its size and frame rate (its bitrate follows from them) + int secondWindowWidth; + int secondWindowHeight; + int secondWindowFps; Language language; CaptureSysKeysMode captureSysKeysMode; GamepadQuitCombo gamepadQuitCombo; @@ -267,6 +296,13 @@ class StreamingPreferences : public QObject void enableYUV444Changed(); void videoDecoderSelectionChanged(); void uiDisplayModeChanged(); + void primaryWindowDisplayChanged(); + void secondWindowDisplayChanged(); + void secondWindowVideoChanged(); + void secondWindowTitleChanged(); + void secondWindowEnabledChanged(); + void secondWindowSourceChanged(); + void secondWindowFullscreenChanged(); void windowModeChanged(); void framePacingChanged(); void connectionWarningsChanged(); diff --git a/app/streaming/secondarystream.cpp b/app/streaming/secondarystream.cpp new file mode 100644 index 0000000000..e8761c14f1 --- /dev/null +++ b/app/streaming/secondarystream.cpp @@ -0,0 +1,369 @@ +#include "secondarystream.h" + +#include "session.h" +#include "streamutils.h" + +#include + +#include + +thread_local int g_DecoderStreamIndex = 0; + +SecondaryStreamWindow::SecondaryStreamWindow(Session* session, const QString& title) + : m_Session(session), + m_Title(title), + m_Window(nullptr), + m_Decoder(nullptr), + m_DecoderLock(SDL_CreateMutex()), + m_VideoFormat(0), + m_VideoWidth(0), + m_VideoHeight(0), + m_FrameRate(0) +{ +} + +SecondaryStreamWindow::~SecondaryStreamWindow() +{ + // The session closes the window on the main thread before it gets here + SDL_assert(m_Window == nullptr); + SDL_DestroyMutex(m_DecoderLock); +} + +int SecondaryStreamWindow::drSetup(int videoFormat, int width, int height, int frameRate, void*, int) +{ + SecondaryStreamWindow* self = Session::get()->m_SecondWindow; + + self->m_VideoFormat = videoFormat; + self->m_VideoWidth = width; + self->m_VideoHeight = height; + self->m_FrameRate = frameRate; + + // Like the first stream, the decoder is created later on the main thread, once the window is shown. + SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION, + "Second video stream is %dx%dx%d (format 0x%x)", + width, height, frameRate, videoFormat); + + return 0; +} + +void SecondaryStreamWindow::drCleanup() +{ + // The decoder and the window belong to the main thread and are destroyed there. +} + +bool SecondaryStreamWindow::open() +{ + if (m_Window != nullptr) { + return true; + } + + // Start at the stream's own size, shrunk to fit inside the display it opens on. The user can resize it. + int width = m_VideoWidth > 0 ? m_VideoWidth : 960; + int height = m_VideoWidth > 0 ? m_VideoHeight : 540; + + std::string title = (m_Session->m_Computer->name + " - " + m_Title).toStdString(); + Uint32 flags = SDL_WINDOW_ALLOW_HIGHDPI | SDL_WINDOW_RESIZABLE; + + // Borderless full screen on the display it opens on, the same state F11 switches to. The window size chosen + // below is what it returns to when F11 is pressed again. + if (m_Session->m_Preferences->secondWindowFullscreen) { + flags |= SDL_WINDOW_FULLSCREEN_DESKTOP; + } + + // Open on the chosen display. Automatic means a display the first window is not on, if there is one, so that + // the two windows do not land on top of each other. + int displayCount = SDL_GetNumVideoDisplays(); + int mainDisplay = m_Session->m_Window != nullptr ? SDL_GetWindowDisplayIndex(m_Session->m_Window) : 0; + int display = -1; + int chosen = m_Session->m_Preferences->secondWindowDisplay; + if (chosen > 0 && chosen <= displayCount) { + display = chosen - 1; + } + else if (chosen > displayCount) { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Display %d was chosen for the second window, but only %d display(s) are attached", + chosen, displayCount); + } + else if (displayCount > 1) { + for (int i = 0; i < displayCount; i++) { + if (i != mainDisplay) { + display = i; + break; + } + } + } + int position = display >= 0 ? SDL_WINDOWPOS_CENTERED_DISPLAY(display) : SDL_WINDOWPOS_UNDEFINED; + + SDL_Rect usable; + if (SDL_GetDisplayUsableBounds(display >= 0 ? display : mainDisplay, &usable) == 0 && usable.w > 0 && usable.h > 0) { + int maxWidth = usable.w * 85 / 100; + int maxHeight = usable.h * 85 / 100; + if (width > maxWidth || height > maxHeight) { + // Keep the stream's shape + if ((long long)width * maxHeight > (long long)height * maxWidth) { + height = height * maxWidth / width; + width = maxWidth; + } + else { + width = width * maxHeight / height; + height = maxHeight; + } + } + } + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, "Second window goes on display %d of %d (first window is on %d)", + display >= 0 ? display + 1 : 0, displayCount, mainDisplay + 1); + + // The first window keeps the keyboard focus and the mouse capture. A second window that takes focus when it + // opens leaves the first one unable to capture, and its mouse events then map onto the second picture. +#ifdef SDL_HINT_WINDOW_NO_ACTIVATION_WHEN_SHOWN + SDL_SetHint(SDL_HINT_WINDOW_NO_ACTIVATION_WHEN_SHOWN, "1"); +#endif + m_Window = SDL_CreateWindow(title.c_str(), position, position, width, height, + flags | StreamUtils::getPlatformWindowFlags()); + if (m_Window == nullptr) { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "SDL_CreateWindow() for the second window failed with platform flags: %s", + SDL_GetError()); + m_Window = SDL_CreateWindow(title.c_str(), position, position, width, height, flags); + } +#ifdef SDL_HINT_WINDOW_NO_ACTIVATION_WHEN_SHOWN + SDL_SetHint(SDL_HINT_WINDOW_NO_ACTIVATION_WHEN_SHOWN, "0"); +#endif + if (m_Window == nullptr) { + SDL_LogError(SDL_LOG_CATEGORY_APPLICATION, + "SDL_CreateWindow() for the second window failed: %s", + SDL_GetError()); + return false; + } + + int actualWidth, actualHeight; + SDL_GetWindowSize(m_Window, &actualWidth, &actualHeight); + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Second window opened: asked for %dx%d, got %dx%d (stream %dx%d)", + width, height, actualWidth, actualHeight, m_VideoWidth, m_VideoHeight); + + return true; +} + +void SecondaryStreamWindow::close() +{ + // The decoder must go before the window it draws into + SDL_LockMutex(m_DecoderLock); + delete m_Decoder; + m_Decoder = nullptr; + SDL_UnlockMutex(m_DecoderLock); + + if (m_Window != nullptr) { + SDL_DestroyWindow(m_Window); + m_Window = nullptr; + } +} + +bool SecondaryStreamWindow::ownsWindow(Uint32 windowId) const +{ + return m_Window != nullptr && SDL_GetWindowID(m_Window) == windowId; +} + +bool SecondaryStreamWindow::recreateDecoder() +{ + StreamingPreferences* prefs = m_Session->m_Preferences; + + // Like the first window: a stream faster than the display cannot be V-synced. + bool enableVsync = prefs->enableVsync; + if (StreamUtils::getDisplayRefreshRate(m_Window) + 5 < m_FrameRate) { + enableVsync = false; + } + + SDL_LockMutex(m_DecoderLock); + delete m_Decoder; + m_Decoder = nullptr; + + // The decoder pulls the frames of the stream it is made for, so tell it before it is created + g_DecoderStreamIndex = 1; + bool ok = Session::chooseDecoder(prefs->videoDecoderSelection, + prefs->rendererSelection, + m_Window, m_VideoFormat, m_VideoWidth, m_VideoHeight, m_FrameRate, + enableVsync, + enableVsync && prefs->framePacing, + false, + m_Decoder); + g_DecoderStreamIndex = 0; + SDL_UnlockMutex(m_DecoderLock); + + if (!ok) { + SDL_LogError(SDL_LOG_CATEGORY_APPLICATION, + "Unable to initialize the decoder of the second window"); + return false; + } + + // Frames that arrived before the decoder existed were dropped, so ask for a keyframe to start from. + LiRequestIdrFrameForStream(1); + return true; +} + +void SecondaryStreamWindow::toggleFullscreen() +{ + if (m_Window == nullptr) { + return; + } + + bool isFullscreen = (SDL_GetWindowFlags(m_Window) & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN_DESKTOP; + if (SDL_SetWindowFullscreen(m_Window, isFullscreen ? 0 : SDL_WINDOW_FULLSCREEN_DESKTOP) != 0) { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Unable to change the fullscreen state of the second window: %s", + SDL_GetError()); + } +} + +void SecondaryStreamWindow::handleKeyEvent(const SDL_KeyboardEvent* event) +{ + // F11 belongs to this window, not to the host. The release is swallowed as well so the host never sees half a key. + if (event->keysym.sym == SDLK_F11 && event->type == SDL_KEYDOWN && event->repeat == 0) { + toggleFullscreen(); + } +} + +void SecondaryStreamWindow::stopAndClose() +{ + // The window is gone for good, so there is no reason for the host to keep encoding what it showed + LiStopVideoStream(1); + close(); +} + +void SecondaryStreamWindow::handleWindowEvent(const SDL_WindowEvent& event) +{ + if (m_Window == nullptr) { + return; + } + + switch (event.event) { + case SDL_WINDOWEVENT_CLOSE: + // The user closed the second window. The session carries on with the first. + stopAndClose(); + break; + + case SDL_WINDOWEVENT_SHOWN: + if (m_Decoder == nullptr && !recreateDecoder()) { + stopAndClose(); + break; + } + + // Whatever the platform did with focus, the stream starts on the first window with the first picture as the + // input target. The user clicks the second window when they want to use it. + if (m_Session->m_Window != nullptr) { + SDL_RaiseWindow(m_Session->m_Window); + } + LiSetInputTargetStream(0); + break; + + case SDL_WINDOWEVENT_SIZE_CHANGED: + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, "Second window size changed to %dx%d", event.data1, event.data2); + if (m_Decoder != nullptr && !(SDL_GetWindowFlags(m_Window) & SDL_WINDOW_MINIMIZED)) { + WINDOW_STATE_CHANGE_INFO info = {}; + info.window = m_Window; + info.stateChangeFlags = WINDOW_STATE_CHANGE_SIZE; + info.width = event.data1; + info.height = event.data2; + if (!m_Decoder->notifyWindowChanged(&info) && !recreateDecoder()) { + stopAndClose(); + } + } + break; + + case SDL_WINDOWEVENT_DISPLAY_CHANGED: + // Another display can have another refresh rate, so start the renderer over. + if (m_Decoder != nullptr && !recreateDecoder()) { + stopAndClose(); + } + break; + + default: + break; + } +} + +void SecondaryStreamWindow::renderFrameOnMainThread() +{ + if (m_Decoder != nullptr) { + m_Decoder->renderFrameOnMainThread(); + } +} + +bool SecondaryStreamWindow::toVideoPosition(int windowX, int windowY, short& x, short& y, short& referenceWidth, short& referenceHeight) +{ + if (m_Window == nullptr || m_VideoWidth <= 0 || m_VideoHeight <= 0) { + return false; + } + + int windowWidth, windowHeight; + SDL_GetWindowSize(m_Window, &windowWidth, &windowHeight); + + SDL_Rect src, dst; + src.x = src.y = 0; + src.w = m_VideoWidth; + src.h = m_VideoHeight; + dst.x = dst.y = 0; + dst.w = windowWidth; + dst.h = windowHeight; + + // The picture is letterboxed inside the window, so map the position into the picture and clamp it there. + StreamUtils::scaleSourceToDestinationSurface(&src, &dst); + + x = (short)qBound(0, windowX - dst.x, dst.w); + y = (short)qBound(0, windowY - dst.y, dst.h); + referenceWidth = (short)dst.w; + referenceHeight = (short)dst.h; + return referenceWidth > 0 && referenceHeight > 0; +} + +void SecondaryStreamWindow::handleMouseMotion(const SDL_MouseMotionEvent* event) +{ + // Ignore the synthetic mouse events that touch input produces + if (event->which == SDL_TOUCH_MOUSEID) { + return; + } + + short x, y, referenceWidth, referenceHeight; + if (toVideoPosition(event->x, event->y, x, y, referenceWidth, referenceHeight)) { + // Positions are mapped onto this window's picture, so tell the host which picture that is first. + LiSetInputTargetStream(1); + LiSendMousePositionEvent(x, y, referenceWidth, referenceHeight); + } +} + +void SecondaryStreamWindow::handleMouseButton(const SDL_MouseButtonEvent* event) +{ + if (event->which == SDL_TOUCH_MOUSEID) { + return; + } + + int button; + switch (event->button) { + case SDL_BUTTON_LEFT: + button = m_Session->m_Preferences->swapMouseButtons ? BUTTON_RIGHT : BUTTON_LEFT; + break; + case SDL_BUTTON_MIDDLE: + button = BUTTON_MIDDLE; + break; + case SDL_BUTTON_RIGHT: + button = m_Session->m_Preferences->swapMouseButtons ? BUTTON_LEFT : BUTTON_RIGHT; + break; + case SDL_BUTTON_X1: + button = BUTTON_X1; + break; + case SDL_BUTTON_X2: + button = BUTTON_X2; + break; + default: + return; + } + + // Put the pointer where the click happened first, so a click does not depend on a motion event before it. + short x, y, referenceWidth, referenceHeight; + if (toVideoPosition(event->x, event->y, x, y, referenceWidth, referenceHeight)) { + LiSetInputTargetStream(1); + LiSendMousePositionEvent(x, y, referenceWidth, referenceHeight); + } + + LiSendMouseButtonEvent(event->type == SDL_MOUSEBUTTONDOWN ? BUTTON_ACTION_PRESS : BUTTON_ACTION_RELEASE, button); +} diff --git a/app/streaming/secondarystream.h b/app/streaming/secondarystream.h new file mode 100644 index 0000000000..f73b69a777 --- /dev/null +++ b/app/streaming/secondarystream.h @@ -0,0 +1,69 @@ +#pragma once + +#include + +#include +#include "SDL_compat.h" +#include "video/decoder.h" + +class Session; + +/** + * The second video stream of a session: a host window shown in a client window of its own. + * + * It owns the SDL window and the decoder for that window. The session creates one when the user asked for a + * second window, lets the connection deliver frames to it through the static callbacks below, and hands it the + * window and pointer events that belong to its window. Everything except the callbacks runs on the session's + * main thread, like the first window. + * + * Absolute pointer input only: the first window can capture the mouse, this one cannot. + */ +class SecondaryStreamWindow +{ +public: + SecondaryStreamWindow(Session* session, const QString& title); + ~SecondaryStreamWindow(); + + // Decoder and renderer callbacks for the connection, for the second stream. The decoder pulls its frames itself, + // the way the first window's does, so there is no submit callback. + static int drSetup(int videoFormat, int width, int height, int frameRate, void* context, int drFlags); + static void drCleanup(); + + // Main thread: show the window. The decoder is created when the window reports that it was shown. + bool open(); + + // Main thread: destroy the decoder and the window. Safe to call twice. + void close(); + + bool isOpen() const { return m_Window != nullptr; } + bool ownsWindow(Uint32 windowId) const; + + void handleWindowEvent(const SDL_WindowEvent& event); + void handleKeyEvent(const SDL_KeyboardEvent* event); + void renderFrameOnMainThread(); + void handleMouseMotion(const SDL_MouseMotionEvent* event); + void handleMouseButton(const SDL_MouseButtonEvent* event); + +private: + // Borderless fullscreen on the display the window is on, and back. A maximized window is not the same: the + // title bar and taskbar take height, so the picture no longer fits the area and gets black bars. + void toggleFullscreen(); + + // Close the window and tell the host to stop sending the stream it showed + void stopAndClose(); + bool recreateDecoder(); + bool toVideoPosition(int windowX, int windowY, short& x, short& y, short& referenceWidth, short& referenceHeight); + + Session* m_Session; + QString m_Title; + + SDL_Window* m_Window; + IVideoDecoder* m_Decoder; + SDL_mutex* m_DecoderLock; + + // Set by drSetup() on the connection thread before the window exists + int m_VideoFormat; + int m_VideoWidth; + int m_VideoHeight; + int m_FrameRate; +}; diff --git a/app/streaming/session.cpp b/app/streaming/session.cpp index 276f6da428..08243b66b9 100644 --- a/app/streaming/session.cpp +++ b/app/streaming/session.cpp @@ -1,4 +1,5 @@ #include "session.h" +#include "secondarystream.h" #include "settings/streamingpreferences.h" #include "streaming/streamutils.h" #include @@ -34,6 +35,7 @@ #define SDL_CODE_GAMECONTROLLER_SET_MOTION_EVENT_STATE 103 #define SDL_CODE_GAMECONTROLLER_SET_CONTROLLER_LED 104 #define SDL_CODE_GAMECONTROLLER_SET_ADAPTIVE_TRIGGERS 105 +#define SDL_CODE_SECOND_STREAM_ENDED 106 #include @@ -66,7 +68,8 @@ CONNECTION_LISTENER_CALLBACKS Session::k_ConnCallbacks = { Session::clRumbleTriggers, Session::clSetMotionEventState, Session::clSetControllerLED, - Session::clSetAdaptiveTriggers + Session::clSetAdaptiveTriggers, + Session::clVideoStreamEnded }; Session* Session::s_ActiveSession; @@ -204,6 +207,23 @@ void Session::clConnectionStatusUpdate(int connectionStatus) } } +void Session::clVideoStreamEnded(int streamIndex, int reason) +{ + SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION, + "Video stream %d ended (reason %d)", + streamIndex, reason); + + // Only the second stream can end on its own. Its window belongs to the main thread, so hand it over. + if (streamIndex == 1) { + SDL_Event event; + event.type = SDL_USEREVENT; + event.user.code = SDL_CODE_SECOND_STREAM_ENDED; + event.user.data1 = nullptr; + event.user.data2 = nullptr; + SDL_PushEvent(&event); + } +} + void Session::clSetHdrMode(bool enabled) { // If we're in the process of recreating our decoder when we get @@ -573,6 +593,7 @@ bool Session::populateDecoderProperties(SDL_Window* window) Session::Session(NvComputer* computer, NvApp& app, StreamingPreferences *preferences) : m_Preferences(preferences ? preferences : StreamingPreferences::get()), m_IsFullScreen(m_Preferences->windowMode != StreamingPreferences::WM_WINDOWED || !WMUtils::isRunningDesktopEnvironment()), + m_SecondWindow(nullptr), m_Computer(computer), m_App(app), m_Window(nullptr), @@ -608,6 +629,8 @@ Session::~Session() destroyUsbForwarding(); SDL_DestroyMutex(m_DecoderLock); + + delete m_SecondWindow; } bool Session::initialize(QQuickWindow* qtWindow) @@ -732,6 +755,65 @@ bool Session::initialize(QQuickWindow* qtWindow) RAND_bytes(reinterpret_cast(m_StreamConfig.remoteInputAesIv), 4); m_StreamConfig.enableMic = m_Preferences->enableMicrophone; + // A second window showing one window of the host. The Settings page saves the title and the command line + // overrides it for a run; for a seat it can also be set under seatprefs// (secondWindow = the window title, and optionally secondWindowWidth, + // secondWindowHeight, secondWindowFps and secondWindowBitrate). Empty means no second window. + { + // The check box in Settings turns the saved title on or off; the command line switches it on for a run + QString secondTitle = m_Preferences->secondWindowEnabled ? m_Preferences->secondWindowTitle : QString(); + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Second window request: title [%s] (settings or command line), host uuid [%s]", + m_Preferences->secondWindowTitle.toUtf8().constData(), + m_Computer->uuid.toUtf8().constData()); + int secondWidth = m_Preferences->secondWindowWidth; + int secondHeight = m_Preferences->secondWindowHeight; + int secondFps = m_Preferences->secondWindowFps; + int secondBitrate = StreamingPreferences::getDefaultBitrate(secondWidth, secondHeight, secondFps, false); + + if (!m_Computer->uuid.isEmpty()) { + QSettings seatSettings; + seatSettings.beginGroup(QString("seatprefs/%1").arg(m_Computer->uuid)); + if (secondTitle.isEmpty()) { + secondTitle = seatSettings.value("secondWindow", QString()).toString(); + } + secondWidth = seatSettings.value("secondWindowWidth", secondWidth).toInt(); + secondHeight = seatSettings.value("secondWindowHeight", secondHeight).toInt(); + secondFps = seatSettings.value("secondWindowFps", secondFps).toInt(); + // A size or rate set for this seat changes the bitrate that fits, unless the seat sets one itself + if (!seatSettings.contains("secondWindowBitrate")) { + secondBitrate = StreamingPreferences::getDefaultBitrate(secondWidth, secondHeight, secondFps, false); + } + secondBitrate = seatSettings.value("secondWindowBitrate", secondBitrate).toInt(); + seatSettings.endGroup(); + } + + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, "Second window final title [%s]", secondTitle.toUtf8().constData()); + + // A whole display needs no title: the host makes it. The name is only the window's title bar. + const bool secondDisplay = m_Preferences->secondWindowEnabled && m_Preferences->secondWindowSource == 1; + if (secondDisplay) { + secondTitle = tr("Second display"); + } + + if (!secondTitle.isEmpty()) { + QByteArray source = secondDisplay ? QByteArray("display:auto") : ("window:" + secondTitle).toUtf8(); + if (source.size() < (int)sizeof(m_StreamConfig.secondVideoSource) && secondWidth > 0 && secondHeight > 0 && + secondFps > 0 && secondBitrate > 0) { + m_StreamConfig.enableSecondVideoStream = true; + memcpy(m_StreamConfig.secondVideoSource, source.constData(), source.size() + 1); + m_StreamConfig.secondVideoWidth = secondWidth; + m_StreamConfig.secondVideoHeight = secondHeight; + m_StreamConfig.secondVideoFps = secondFps; + m_StreamConfig.secondVideoBitrateKbps = secondBitrate; + m_SecondWindow = new SecondaryStreamWindow(this, secondTitle); + } + else { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Ignoring the second window: the title is too long or the size, frame rate or bitrate is invalid"); + } + } + } + switch (m_Preferences->audioConfig) { case StreamingPreferences::AC_STEREO: @@ -1390,6 +1472,18 @@ void Session::getWindowDimensions(int& x, int& y, } } + // The user may have chosen the display for this window + if (m_Window == nullptr && m_Preferences->primaryWindowDisplay > 0) { + if (m_Preferences->primaryWindowDisplay <= SDL_GetNumVideoDisplays()) { + displayIndex = m_Preferences->primaryWindowDisplay - 1; + } + else { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "Display %d was chosen for the stream window, but only %d display(s) are attached", + m_Preferences->primaryWindowDisplay, SDL_GetNumVideoDisplays()); + } + } + // Get the usable display bounds SDL_Rect usableBounds; if (SDL_GetDisplayUsableBounds(displayIndex, &usableBounds) != 0) { @@ -1745,6 +1839,20 @@ bool Session::startConnectionAsync() false); } + if (m_SecondWindow != nullptr) { + // The second window's decoder pulls its own stream, exactly as the first window's does with its stream, so it + // has the same capabilities and no submit callback. + LiInitializeVideoCallbacks(&m_VideoCallbacks2); + m_VideoCallbacks2.setup = SecondaryStreamWindow::drSetup; + m_VideoCallbacks2.cleanup = SecondaryStreamWindow::drCleanup; + m_VideoCallbacks2.capabilities = m_VideoCallbacks.capabilities; + LiSetSecondVideoStreamCallbacks(&m_VideoCallbacks2, nullptr, 0); + } + + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, "Starting connection: second window %s, second stream %s", + m_SecondWindow != nullptr ? "set" : "not set", + m_StreamConfig.enableSecondVideoStream ? "requested" : "not requested"); + int err = LiStartConnection(&hostInfo, &m_StreamConfig, &k_ConnCallbacks, &m_VideoCallbacks, &m_AudioCallbacks, NULL, 0, NULL, 0); @@ -2159,6 +2267,18 @@ void Session::exec() // Switch to async logging mode when we enter the SDL loop StreamUtils::enterAsyncLoggingMode(); + // Show the second window, if the host agreed to send the second stream + if (m_SecondWindow != nullptr) { + if (!LiIsSecondVideoStreamActive()) { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "The second window was requested, but the host did not start a second video stream"); + } + else if (!m_SecondWindow->open()) { + SDL_LogWarn(SDL_LOG_CATEGORY_APPLICATION, + "The second window could not be opened; continuing with one"); + } + } + // Hijack this thread to be the SDL main thread. We have to do this // because we want to suspend all Qt processing until the stream is over. SDL_Event event; @@ -2208,6 +2328,18 @@ void Session::exec() if (m_VideoDecoder != nullptr) { m_VideoDecoder->renderFrameOnMainThread(); } + // The event does not say which stream the frame is for, and a decoder with nothing queued + // does nothing, so give both a turn. + if (m_SecondWindow != nullptr) { + m_SecondWindow->renderFrameOnMainThread(); + } + break; + case SDL_CODE_SECOND_STREAM_ENDED: + // The host ended the second stream (for example its window was closed). The session carries on. + if (m_SecondWindow != nullptr) { + m_SecondWindow->close(); + LiSetInputTargetStream(0); + } break; case SDL_CODE_FLUSH_WINDOW_EVENT_BARRIER: m_FlushingWindowEventsRef--; @@ -2243,8 +2375,9 @@ void Session::exec() break; case SDL_WINDOWEVENT: - // Early handling of some events - switch (event.window.event) { + // Early handling of some events. Focus and pointer-leave belong to the first window: if the second + // window's focus gain re-captured the mouse, the capture would confine the pointer to the second window. + switch (m_SecondWindow != nullptr && m_SecondWindow->ownsWindow(event.window.windowID) ? 0 : event.window.event) { case SDL_WINDOWEVENT_FOCUS_LOST: if (m_Preferences->muteOnFocusLoss) { m_AudioMuted = true; @@ -2264,6 +2397,21 @@ void Session::exec() presence.runCallbacks(); + // Events of the second window are its own business; they must not recreate the first window's decoder + if (m_SecondWindow != nullptr && m_SecondWindow->ownsWindow(event.window.windowID)) { + m_SecondWindow->handleWindowEvent(event.window); + break; + } + + // With a second window open, closing the first one no longer ends the program by itself (SDL only + // quits when the last window closes), so treat it as the quit it has always been. + if (m_SecondWindow != nullptr && event.window.event == SDL_WINDOWEVENT_CLOSE && + event.window.windowID == SDL_GetWindowID(m_Window)) { + SDL_LogInfo(SDL_LOG_CATEGORY_APPLICATION, + "The first window was closed"); + goto DispatchDeferredCleanup; + } + // Capture the mouse on SDL_WINDOWEVENT_ENTER if needed if (needsFirstEnterCapture && event.window.event == SDL_WINDOWEVENT_ENTER) { m_InputHandler->setCaptureActive(true); @@ -2449,14 +2597,33 @@ void Session::exec() case SDL_KEYUP: case SDL_KEYDOWN: presence.runCallbacks(); + if (m_SecondWindow != nullptr && m_SecondWindow->ownsWindow(event.key.windowID) && + event.key.keysym.sym == SDLK_F11) { + m_SecondWindow->handleKeyEvent(&event.key); + break; + } m_InputHandler->handleKeyEvent(&event.key); break; case SDL_MOUSEBUTTONDOWN: case SDL_MOUSEBUTTONUP: presence.runCallbacks(); + if (m_SecondWindow != nullptr) { + if (m_SecondWindow->ownsWindow(event.button.windowID)) { + m_SecondWindow->handleMouseButton(&event.button); + break; + } + LiSetInputTargetStream(0); + } m_InputHandler->handleMouseButtonEvent(&event.button); break; case SDL_MOUSEMOTION: + if (m_SecondWindow != nullptr) { + if (m_SecondWindow->ownsWindow(event.motion.windowID)) { + m_SecondWindow->handleMouseMotion(&event.motion); + break; + } + LiSetInputTargetStream(0); + } m_InputHandler->handleMouseMotionEvent(&event.motion); break; case SDL_MOUSEWHEEL: @@ -2547,6 +2714,13 @@ void Session::exec() m_VideoDecoder = nullptr; SDL_UnlockMutex(m_DecoderLock); + // The second window's decoder and window go too, for the same reason. + // The object itself stays until the session is destroyed: the connection's + // callbacks may still run until LiStopConnection(). + if (m_SecondWindow != nullptr) { + m_SecondWindow->close(); + } + // Propagate state changes from the SDL window back to the Qt window // // NB: We're making a conscious decision not to propagate the maximized diff --git a/app/streaming/session.h b/app/streaming/session.h index c4ca682e4e..17f130b925 100644 --- a/app/streaming/session.h +++ b/app/streaming/session.h @@ -12,6 +12,7 @@ #include "video/overlaymanager.h" class MicrophoneCapture; +class SecondaryStreamWindow; namespace UsbForwarding { class Manager; @@ -100,6 +101,7 @@ class Session : public QObject Q_OBJECT friend class SdlInputHandler; + friend class SecondaryStreamWindow; friend class DeferredSessionCleanupTask; friend class AsyncConnectionStartThread; @@ -239,6 +241,9 @@ class Session : public QObject static void clSetAdaptiveTriggers(uint16_t controllerNumber, uint8_t eventFlags, uint8_t typeLeft, uint8_t typeRight, uint8_t *left, uint8_t *right); + static + void clVideoStreamEnded(int streamIndex, int reason); + static int arInit(int audioConfiguration, const POPUS_MULTISTREAM_CONFIGURATION opusConfig, @@ -264,6 +269,8 @@ class Session : public QObject SupportedVideoFormatList m_SupportedVideoFormats; // Sorted in order of descending priority STREAM_CONFIGURATION m_StreamConfig; DECODER_RENDERER_CALLBACKS m_VideoCallbacks; + DECODER_RENDERER_CALLBACKS m_VideoCallbacks2; + SecondaryStreamWindow* m_SecondWindow; AUDIO_RENDERER_CALLBACKS m_AudioCallbacks; NvComputer* m_Computer; NvApp m_App; diff --git a/app/streaming/video/ffmpeg.cpp b/app/streaming/video/ffmpeg.cpp index 337c6e72e0..d05850a754 100644 --- a/app/streaming/video/ffmpeg.cpp +++ b/app/streaming/video/ffmpeg.cpp @@ -225,6 +225,9 @@ enum AVPixelFormat FFmpegVideoDecoder::ffGetFormat(AVCodecContext* context, return AV_PIX_FMT_NONE; } +// Set by the second window around the creation of its decoder. 0 means the first video stream. +extern thread_local int g_DecoderStreamIndex; + FFmpegVideoDecoder::FFmpegVideoDecoder(bool testOnly) : m_Pkt(av_packet_alloc()), m_VideoDecoderCtx(nullptr), @@ -244,7 +247,8 @@ FFmpegVideoDecoder::FFmpegVideoDecoder(bool testOnly) m_NeedsSpsFixup(false), m_TestOnly(testOnly), m_CurrentTestMode(TestMode::TestFrameOnly), - m_DecoderThread(nullptr) + m_DecoderThread(nullptr), + m_StreamIndex(g_DecoderStreamIndex) { SDL_zero(m_ActiveWndVideoStats); SDL_zero(m_LastWndVideoStats); @@ -277,7 +281,7 @@ void FFmpegVideoDecoder::reset() // It might be touching things we're about to free. if (m_DecoderThread != nullptr) { SDL_AtomicSet(&m_DecoderThreadShouldQuit, 1); - LiWakeWaitForVideoFrame(); + LiWakeWaitForVideoFrameEx(m_StreamIndex); SDL_WaitThread(m_DecoderThread, NULL); SDL_AtomicSet(&m_DecoderThreadShouldQuit, 0); m_DecoderThread = nullptr; @@ -1873,7 +1877,7 @@ void FFmpegVideoDecoder::decoderThreadProc() // Waiting for input. All output frames have been received. // Block until we receive a new frame from the host. - if (!LiWaitForNextVideoFrame(&handle, &du)) { + if (!LiWaitForNextVideoFrameEx(m_StreamIndex, &handle, &du)) { // This might be a signal from the main thread to exit continue; } @@ -2063,7 +2067,7 @@ void FFmpegVideoDecoder::decoderThreadProc() // No output data, so let's try to submit more input data, // while we're waiting for this to frame to come back. - if (LiPollNextVideoFrame(&handle, &du)) { + if (LiPollNextVideoFrameEx(m_StreamIndex, &handle, &du)) { // FIXME: Handle EAGAIN on avcodec_send_packet() properly? LiCompleteVideoFrame(handle, submitDecodeUnit(du)); } @@ -2097,7 +2101,12 @@ void FFmpegVideoDecoder::decoderThreadProc() // Just in case the error resulted in the loss of the frame, // request an IDR frame to reset our decoder state. - LiRequestIdrFrame(); + if (m_StreamIndex == 0) { + LiRequestIdrFrame(); + } + else { + LiRequestIdrFrameForStream(m_StreamIndex); + } } } while (err == AVERROR(EAGAIN) && !SDL_AtomicGet(&m_DecoderThreadShouldQuit)); diff --git a/app/streaming/video/ffmpeg.h b/app/streaming/video/ffmpeg.h index 1e9e9d1851..42f7f98050 100644 --- a/app/streaming/video/ffmpeg.h +++ b/app/streaming/video/ffmpeg.h @@ -132,6 +132,7 @@ class FFmpegVideoDecoder : public IVideoDecoder { TestMode m_CurrentTestMode; SDL_Thread* m_DecoderThread; SDL_atomic_t m_DecoderThreadShouldQuit; + int m_StreamIndex; // which video stream this decoder pulls: 0 = first window, 1 = second window // Data buffers in the queued DU are not valid QQueue m_FrameInfoQueue; diff --git a/moonlight-common-c/moonlight-common-c b/moonlight-common-c/moonlight-common-c index bd3443c790..02525dbdae 160000 --- a/moonlight-common-c/moonlight-common-c +++ b/moonlight-common-c/moonlight-common-c @@ -1 +1 @@ -Subproject commit bd3443c790d07597f927d5a3d1560ca107a3ce22 +Subproject commit 02525dbdae059a89a6138a4362a0f2be66ffb9c6