StreamMatrix

StreamMatrix is a high-performance, low-latency video matrix application designed for monitoring multiple live RTSP, RTMP, HTTP, and local video feeds simultaneously. Built with Qt Quick (QML) and FFmpeg, it is specifically tuned for optimal resource efficiency on desktop Linux and embedded systems like the Raspberry Pi 4 and Raspberry Pi 5.

---

🌟 Acknowledgement & Origins

StreamMatrix is an independent, modernized fork of the wonderful CCTV Viewer project originally created by Evgeny S. Maksimov (@iEvgeny).

We extend our sincere gratitude to Evgeny for establishing the foundational architecture and QML AV pipeline. As upstream development slowed down, this fork was created to:

• Actively maintain and modernize the codebase for newer Linux environments (Debian 12/13 Trixie, Raspberry Pi OS).
• Deliver native support for modern Wayland compositors (labwc, wayfire) and X11.
• Optimize multi-stream hardware acceleration (DRM, VA-API) to allow smooth, low-power playback of high-resolution IP camera grids on single-board computers like the Raspberry Pi 5.
• Introduce critical reliability features including stream health diagnostics, automated reconnection watchdog, and dual-profile sub-stream management.

---

🚀 Key Features & Enhancements

⚡ Performance & Hardware Acceleration

• Raspberry Pi 5 & 4 Tuned: Seamless out-of-the-box hardware decoding (-hwaccel drm) and smooth Wayland/Labwc rendering.
• Cached SwsContext Scaling: Eliminates per-frame memory allocation/deallocation churn and scaling bottleneck, reducing CPU load.
• Zero-Copy Video Path: Improved GLX and DRM texture handling with dynamic resolution change support.
• Thread-Safe Glitch-Free Audio: Overhauled audio I/O pipeline with continuous frame buffering and thread synchronization, eliminating audio stuttering and buffer underruns.
• Deadlock-Free Shutdown: Interrupt-aware producer/consumer queues prevent deadlocks and hung processes on exit.

📊 Diagnostics & Stream Health

• Live Stream Diagnostics HUD: Press D to toggle an on-screen HUD displaying real-time FPS, Resolution, Video Codec, Hardware Acceleration status [HW], and Bitrate.
• Interactive Reconnection & Retry Status Screen:
  • Tactile Button Confirmation: Clicking the "Retry Now" button provides instantaneous visual feedback—transforming into an active "Retrying..." state with an animated spinner, debounced to prevent duplicate reconnection storms.
  • Live Progress & Backoff Countdown: The reconnect overlay keeps you informed at every stage, showing active connection attempts ("Connecting (attempt N)...", "Attempting connection to stream...") and a real-time second-by-second countdown ("Retrying in Xs (attempt N)...") during exponential backoff.
  • Detailed Connection Error Diagnostics: Unveils the exact root cause behind feed failures directly from FFmpeg (e.g., "Error: Connection refused", "Error: Connection timed out", "Error: Server returned 404 Not Found", "Error: No route to host").
• Auto-Reconnect Watchdog: Automatically detects dropped feeds or frozen video streams (watchdog triggers after 7s without frames) and executes exponential backoff recovery without freezing the UI or blocking worker threads.
• Dual-Profile Resolution Switching: Configure a high-resolution main stream (URL) and low-bandwidth sub-stream (Sub-Stream URL) per camera. The grid automatically renders lightweight sub-streams and seamlessly elevates to full resolution upon double-clicking into full-screen.

🖥️ Modernized UI & System Integration

• Unified Blue Branding: Clean, modern blue application icon and UI accents designed for high contrast and clarity on both desktop Linux and wall-mounted monitoring displays.
• Quote-Aware FFmpeg Parser: Tokenizer supports complex arguments and quoted options (e.g. -rtsp_transport tcp -fflags nobuffer).
• Multi-User Safe: Instance locking uses user-isolated runtime paths ($XDG_RUNTIME_DIR), allowing multiple users to run isolated instances concurrently.
• Smart Wayland/X11 Launcher: Automatically detects active display sockets when launched from desktop shortcuts, systemd services, or SSH.
• Customizable Layouts & Presets: Flexible $M \times N$ grid layouts with span adjustments, digital pan/zoom, preset carousel cycling, and kiosk mode.

---

⌨️ Keyboard Shortcuts

Shortcut	Action
F11 / Double Click	Toggle Full-Screen Mode
D	Toggle Stream Diagnostics HUD (FPS, Bitrate, Codec, Resolution)
M	Toggle Mute/Unmute for Selected Viewport
Alt + 1 .. 9	Switch Directly to Preset 1 through 9
Alt + ← / →	Switch to Previous / Next Preset
Space	Pause / Resume Preset Carousel
Esc	Exit Full-Screen or Deselect
Ctrl + Q	Quit Application

---

🛠️ Building & Installing

Prerequisites (Debian / Ubuntu / Raspberry Pi OS)

sudo apt update
sudo apt install -y \
    build-essential \
    cmake \
    pkg-config \
    qtbase5-dev \
    qtdeclarative5-dev \
    qtmultimedia5-dev \
    qttools5-dev \
    qttools5-dev-tools \
    qml-module-qtquick2 \
    qml-module-qtquick-layouts \
    qml-module-qtquick-controls2 \
    qml-module-qtquick-window2 \
    qml-module-qtmultimedia \
    qml-module-qtgraphicaleffects \
    libavformat-dev \
    libavcodec-dev \
    libavutil-dev \
    libswscale-dev \
    libswresample-dev \
    libavdevice-dev \
    libva-dev

Clone & Build

Clone the repository recursively to fetch all dependencies:

git clone --recurse-submodules https://github.com/ldl805/stream-matrix.git
cd stream-matrix

# Configure and build
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

Install System-Wide

sudo cmake --install build

Or run directly from the build directory using the included launcher:

./run.sh

Flatpak Build & Run

To build and run StreamMatrix as a universal, sandboxed Flatpak package:

# 1. Install flatpak and flatpak-builder
sudo apt install -y flatpak flatpak-builder

# 2. Add Flathub and install the KDE 5.15 runtime & SDK
flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo
flatpak install -y flathub org.kde.Platform//5.15-25.08 org.kde.Sdk//5.15-25.08

# 3. Build and install locally
flatpak-builder --user --install --force-clean build-flatpak org.streammatrix.StreamMatrix.yaml

# 4. Run StreamMatrix
flatpak run org.streammatrix.StreamMatrix

---

⚙️ Configuration & Hardware Acceleration

Recommended RTSP FFmpeg Options

In Settings → Viewport → Default FFmpeg options, the following defaults are pre-configured for low latency and robust streaming:

-rtsp_transport tcp -fflags nobuffer  -timeout 5000000 -probesize 500000 -analyzeduration 1000000

(Note for Raspberry Pi 5: The BCM2712 CPU handles H.264 streams using multi-threaded NEON software decode; hardware decoding on Pi 5 is reserved for HEVC/H.265. Omit -hwaccel drm for H.264 feeds).

Common Camera RTSP Formats

• Main Stream: rtsp://username:password@192.168.1.50:554/stream1
• Sub Stream: rtsp://username:password@192.168.1.50:554/stream2

---

📜 License & Credits

• StreamMatrix is licensed under the GNU General Public License v3.0.
• Based on cctv-viewer © Evgeny S. Maksimov.
• Uses FFmpeg (LGPL/GPL) and the Qt Framework (LGPLv3/GPLv3).