Skip to content

fix(health): keep files with unrepaired known holes degraded after a clean sweep - #1053

Open
firestaerter3 wants to merge 2 commits into
kipsilabs:mainfrom
firestaerter3:fix/known-holes-keep-degraded
Open

firestaerter3 wants to merge 2 commits into
kipsilabs:mainfrom
firestaerter3:fix/known-holes-keep-degraded

Conversation

@firestaerter3

Copy link
Copy Markdown
Contributor

Fixes #1052

A clean health sweep marked a file healthy without looking at its persisted hole map, so a file that was zero-filled during playback went back to healthy while the holes were still there (details in the issue).

Changes

  • When a sweep finds nothing missing, judgeValidation now reads the file's metadata again and checks known_holes. Holes that no local PAR2 patch covers keep the file degraded, with error type known_holes. This goes through the existing degraded handling, so PAR2 repair is queued as for any degraded file.
  • The verdict is never failed. The acceptable-missing threshold and the repair flow are unchanged.
  • Holes count regardless of age or provider fingerprint. Those only decide whether playback re-fetches before padding, and a STAT sweep cannot show that a missing body is back.
  • The read happens at verdict time, not when the check is prepared, so a hole recorded while the sweep runs is not missed.
  • If that read fails, or the metadata is gone, the result is inconclusive (pending, retried later, no retry used) instead of healthy.
  • A file that is not eligible for hole classification (encrypted, non-video) still becomes healthy as before.
  • The loop that skips patched segments moves into a helper shared with classifyHoles.
  • content_verify_test.go and content_verify_report_test.go now give their checker a metadata service, since the clean path reads metadata.

Known limits

  • A hole is never removed from known_holes when a later fetch of that article succeeds. So a file whose provider has refilled the article stays degraded until PAR2 patches it or the record is reset. Removing a hole on a successful fetch would be a separate change in nzbfilesystem.
  • There is still a very short window between reading the hole map and writing the healthy result. A hole recorded exactly then can still be missed. Closing that fully needs a version check in the metadata layer, which seemed too much for this fix.

Testing

  • New tests, which return file_healthy without the fix:
    • sampled and full sweeps through CheckFile and CheckFilesBatch;
    • a stale hole from another provider set;
    • a hole added between prepare and verdict;
    • unreadable and deleted metadata;
    • the worker in batch and direct mode.
  • Unchanged behaviour is also covered: a patched hole and a file with no holes become healthy, and an ineligible file with a clean sweep stays healthy.
  • go vet and go test -race ./internal/health/... ./internal/holes/... ./internal/metadata/... pass.
  • Not linted, and the frontend was not built; no frontend files change.

firestaerter3 and others added 2 commits November 5, 2026 19:17
…clean sweep

A hole padded during playback is persisted in the file's known_holes and
the file is marked degraded. The next health check samples with STAT, and
a provider can answer STAT for an article whose body is gone, so the sweep
finds nothing missing. judgeValidation then returned file_healthy without
looking at the hole map, and the file went back to healthy on the long
schedule while the hole was still there.

A clean sweep now checks the persisted holes. Holes not covered by a local
PAR2 patch keep the file degraded (error type known_holes), which also
queues PAR2 repair through the existing degraded path. The verdict is
never failed, so the acceptable-missing threshold and the repair flow are
unchanged. Holes count regardless of age or provider fingerprint: those
only decide whether playback re-fetches before padding, and nothing in a
STAT sweep can disprove a missing body.

The patch-exclusion loop moves into a helper shared with classifyHoles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ad file healthy

Two gaps in the clean-sweep known-holes check:

- It only looked for holes when prepareCheck had seen a non-empty hole map.
  Playback can record a hole while the STAT sweep runs, and that file was
  then still marked healthy. The verdict now always re-reads metadata on a
  clean sweep. That is one extra metadata read per clean check; health
  checks run per file on a schedule of hours to months, so the cost is
  small next to the network sweep itself.
- A failed re-read was treated as "no holes", so a transient read error
  produced file_healthy. It now makes the check inconclusive, the same
  outcome as an unfinished STAT sweep: the prior status is kept, the next
  check is pushed out, and no retry is consumed. Metadata that is gone by
  the verdict is handled the same way; the next check's prepare step finds
  it missing and removes the record. An ineligible file whose metadata
  reads fine still becomes healthy as before.

The content-verification tests built a checker without a metadata service;
they now get one holding metadata for the files they judge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@javi11

javi11 commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

@firestaerter3 test are failing can you fix it?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Health check marks a zero-filled file healthy again while its known holes remain

2 participants