Skip to content

orbiscreen | fix: drop stale frames and eliminate stream latency accumulation (#77) - #78

Merged
shadow-x78 merged 1 commit into
shadow-x78:mainfrom
Yashb404:fix/stream-latency-accumulation
Sep 14, 2026
Merged

shadow-x78 merged 1 commit into
shadow-x78:mainfrom
Yashb404:fix/stream-latency-accumulation

Conversation

@Yashb404

Copy link
Copy Markdown
Contributor

What does this PR do?

Resolves issue #77 by stopping the video stream from buffering stale frames and accumulating seconds of video and mouse latency over time (both via Wi-Fi and USB Direct/AOA).

Key changes:

  1. HTTP MPEG-TS Pipeline Tuning:
    • Changed appsink in crates/orbiscreen-transport/src/lib.rs from drop=false max-buffers=16 to drop=true max-buffers=1 sync=false.
    • Set is-live=true do-timestamp=true on the pipeline so that under network or USB backpressure, the sink immediately drops stale frames rather than queueing up multi-second delays.
  2. PTS Timeline Continuity:
    • Clamped delta PTS calculations in stream_handler across frame gaps exceeding 250ms back to the nominal frame time (e.g. 16.6ms for 60fps), preventing timeline desync and buffer inflation in mpegtsmux and client decoders after laptop suspend/resume or heavy pipeline stalls.
  3. USB AOA Transport Buffer Optimization:
    • Reduced sync_channel capacity in crates/orbiscreen-transport/src/aoa.rs from 64 chunks (1MB) to 8 chunks, eliminating multi-frame host queueing over USB.
  4. Multi-Threaded Color Conversion:
    • Added n-threads=4 to CPU videoconvert elements across capture and encode pipelines (kwin_virtual.rs, wayland.rs, and orbiscreen-encode/lib.rs) to prevent single-core bottlenecks when converting BGRA to NV12 at 1080p+.
  5. Appsrc / Appsink Constraints:
    • Constrained appsrc max-bytes to 1 uncompressed frame and encoder appsink max-buffers to 1 across capture pipelines to prevent buffer bloat.

Why?

Addresses Issue #77 ([bug] Video stream accumulates seconds of latency instead of dropping stale frames (Wi-Fi and USB/AOA)) and the pending CHANGELOG.md entry under [Unreleased].

Prior to this fix, moving windows continuously, dropping packets, or resuming from system sleep caused frames to queue up indefinitely inside appsink (max-buffers=16, drop=false) and the AOA sync channel. The display on the client fell behind real time by 5–15+ seconds ("slow motion" effect). With zero-delay dropping and clamped timestamp deltas, the stream remains locked to real time.

How was it tested?

  • cargo fmt --all
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • cargo test --workspace
  • Manual smoke test on: Ubuntu 24.04 LTS / KDE Plasma 6 (Wayland) + AMD 680M / NVIDIA RTX 3050, connected to Android tablet over USB Direct (AOA) and Wi-Fi. Verified that window dragging and rapid pointer movement no longer accumulate latency or rubber-band.

Checklist

  • No new warnings introduced.
  • File headers match Orbiscreen style (// Orbiscreen - <module> (GPL-3.0-or-later) + GitHub URL).
  • CHANGELOG.md updated.

@Yashb404

Copy link
Copy Markdown
Contributor Author

Allowing it to drop frames and reducing the buffer has genuinely shown me a lot of improvement on my personal setup. The queue just created a bottleneck in my setup especially if the hardware is on the slower side. The performance is comparable to Spacedesk now, especially on my decade old tablet.

Opened a pull request if these suggestions can help out the project as a whole.

I was looking around and found some other minor improvements as well. Would love a discussion space to discuss them before opening a PR.

@shadow-x78

shadow-x78 commented Sep 14, 2026

Copy link
Copy Markdown
Owner

@Yashb404 Thanks for PR and help, really glad it helped with your tablet.

I'll check PR in a bit. Setting up a Discord server soon to talk through new ideas, and I'll link it in README once it's up.

@shadow-x78
shadow-x78 self-requested a review September 14, 2026 16:33
@shadow-x78 shadow-x78 self-assigned this Sep 14, 2026
@shadow-x78
shadow-x78 removed their request for review September 14, 2026 16:34
@shadow-x78
shadow-x78 merged commit 9b6fae0 into shadow-x78:main Sep 14, 2026
2 checks passed
shadow-x78 added a commit that referenced this pull request Sep 14, 2026
@Yashb404
Yashb404 deleted the fix/stream-latency-accumulation branch September 15, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants