Skip to content

fix(video): stream canonical remux with bounded memory (#580) - #582

Closed
shobhitagnihotri69 wants to merge 1 commit into
Hebbian-Robotics:mainfrom
shobhitagnihotri69:fix/580-video-bounded-memory
Closed

shobhitagnihotri69 wants to merge 1 commit into
Hebbian-Robotics:mainfrom
shobhitagnihotri69:fix/580-video-bounded-memory

Conversation

@shobhitagnihotri69

@shobhitagnihotri69 shobhitagnihotri69 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #580. Eliminates full-video memory materialization in Episode.video() by streaming canonical H.264 access units directly into ffmpeg stdin with bounded memory.

Key Changes

  • Lazy Batch Streaming: Replaced self.channel(topic) and b"".join(units) with self.iter_decoded_batches().
  • Bounded Exact Median FPS: Implemented estimate_fps_from_streaming_log_times backed by a temporary SQLite table to preserve the exact delta median without buffering millions of timestamps in memory.
  • Incremental B-Frame Validation: Introduced _PictureCodingScanner to scan slice headers access-unit by access-unit during streaming.
  • Robust Process Management: Stderr redirected to a temporary file to avoid pipe deadlocks; guaranteed subprocess termination and cleanup on early exits or broken pipes.
  • Comprehensive Regressions: Added tests/test_video_streaming.py validating incremental consumption, process cleanup, and error precedence.

Verification

  • uv run pytest tests/test_video.py tests/test_video_streaming.py: 52 passed in 11.11s.
  • uv run ruff check src tests: All checks passed.

@greptile-apps

greptile-apps Bot commented Sep 21, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no actionable correctness or security defects were identified.

Summary

Changes canonical video remuxing to consume bounded batches and stream access units into ffmpeg instead of materializing the entire camera channel and concatenated video payload.

  • Computes exact median-based FPS using a temporary disk-backed interval index.
  • Classifies picture coding types incrementally while retaining B-frame rejection.
  • Preserves atomic output replacement and adds subprocess cleanup, failure-path, batching, and cache-reuse coverage.
  • No actionable defects were identified.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Episode.video] --> B{Cached FPS and MP4 exist?}
  B -->|Yes| C[Return completed MP4]
  B -->|No| D[Read timestamp batches]
  D --> E[Compute exact median using temporary SQLite index]
  E --> F{MP4 exists?}
  F -->|Yes| C
  F -->|No| G[Read and decode bounded batches]
  G --> H[Scan access units and stream to ffmpeg]
  H --> I{Validation and remux succeed?}
  I -->|Yes| J[Atomically replace output]
  J --> C
  I -->|No| K[Reap process and remove temporary output]
Loading

Reviews (1) · Last reviewed commit: "fix(video): stream canonical remux with ..."

@kstonekuan

Copy link
Copy Markdown
Contributor

Closing on first-PR-wins: #581 opened 44 minutes earlier and is in review.

For the record, this branch's tree is byte-identical to #581's across all four files, so there is nothing here that merging #581 would not already deliver. If that is not what you expected, say so and I will take another look.

Open good first issues: https://github.com/Hebbian-Robotics/hflow/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22

@kstonekuan kstonekuan closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Episode.video() materializes the full camera stream multiple times before ffmpeg remux

2 participants