Skip to content

fix(camera_video): stage scratch-bound artifacts to durable artifacts/ directory - #548

Closed
amansingh-121 wants to merge 1 commit into
Hebbian-Robotics:mainfrom
amansingh-121:fix/camera-video-scratch-artifacts
Closed

amansingh-121 wants to merge 1 commit into
Hebbian-Robotics:mainfrom
amansingh-121:fix/camera-video-scratch-artifacts

Conversation

@amansingh-121

Copy link
Copy Markdown
Contributor

Fixes #545.

Problem

When camera_video (and other enrichments) produce artifacts in the scratch/ directory, the publish loop was treating them as durable. The scratch directory is cleaned on SYNC stage, so artifacts published to scratch become stale or missing after a reprocess. This breaks the evidence chain - users opening the artifact URI get wrong pixels or a 404 with no indication of the problem.

Root Cause

In app.py, the artifact publishing logic checked if an artifact was under run_dir and, if so, used its relative path directly as the artifact key. Since scratch is inside run_dir, files in scratch were published without any consideration that they would be deleted on the next run.

The comment at test_camera_video.py:66 (assert published_mp4.is_relative_to(data_root)) was flawed because it passed even when the artifact was in scratch (paradoxically, scratch is inside data_root in the test setup).

Solution

In the publish loop (app.py), add a check to detect if an artifact is under scratch_dir. If so:

  1. Copy the artifact to a stable run_dir/artifacts/<step>/ location
  2. Publish from that staged copy using the same key format

This ensures the published URI remains valid across reprocess cycles.

Changes

  • src/hflow/app.py: Add scratch detection and staging logic before publishing
  • tests/test_camera_video.py: Update assertion to verify artifacts are NOT under scratch

Testing

  • ruff check and ruff format --check pass
  • Note: Full test suite cannot be run locally on Windows due to fcntl import and missing ffmpeg, but CI coverage is identical to the existing test setup

Definition of Done Met

  1. ✅ Artifacts produced in scratch are staged to permanent location
  2. ✅ Published URI persists across reprocess
  3. ✅ Test assertion corrected (currently checks "scratch" not in published_mp4.parts)

…/ directory

When camera_video (and other enrichments) produce artifacts in the scratch
directory, the publish loop was treating them as durable. On reprocess,
scratch is cleaned and these artifacts become stale or missing.

Now the publish loop detects if an artifact is under scratch_dir and copies
it to run_dir/artifacts/ before publishing, ensuring the URI remains valid
across reprocess cycles.

Also updates the test assertion in test_camera_video.py to verify that
published artifacts are NOT under scratch.
@github-actions

Copy link
Copy Markdown

👋 Hi @amansingh-121 — thanks for the contribution! To keep starter issues available
for other contributors and give every pull request a real review, we accept
1 open pull request per contributor at a time.

You already have #542 open, so this one is being closed automatically.
Once your open pull request is merged or closed, feel free to reopen this one —
no work is lost.

@github-actions github-actions Bot closed this Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: camera_video publishes scratch-bound artifact URIs as durable, causing silent byte-replacement or hard deletion on sync rewrite

1 participant