borg2 check --archives-only crashes if a pack referenced by the index is missing #10466

borg2 master (c195377): if a pack that the chunks index references is missing from packs/, borg check --archives-only without --repair crashes with a traceback (rc 17) instead of reporting the lost objects.

The full borg check is fine: the repository check detects the missing pack (#10069), and Repository.check() removes the entries of missing packs from the session chunk index (remove_missing_pack_entries()), so the archives check that runs afterwards sees these chunks as missing and reports them normally ("Archive metadata block ... is missing!", etc.). --archives-only --repair is also fine, because --repair rebuilds the index from the real packs.

With --archives-only and no --repair, ArchiveChecker.check() uses the loaded index as-is (archive.py#L2328), which still points into the missing pack. The first read of such a chunk raises Repository.ObjectNotFound (get()) or Repository.PackNotFound (get_many()), and nobody catches it, because the archives check only handles IntegrityErrorBase at these places.

Reproduce

borg -r repo repo-create -e aes256-ocb
head -c 40000000 /dev/urandom > data/f1   # f1..f3, so there are several packs
borg -r repo create a1 data
borg -r repo create a2 data
rm repo/packs/xx/<pack>                   # a pack holding an archive metadata object, see below
borg -r repo check --archives-only

Results (repo with 5 packs; which pack gets removed decides the crash site):

Crash sites (master c195377)

• rebuild_archives(): cdata = self.repository.get(archive_id) for the archive metadata object, archive.py#L2826 (only a key absent from the index is handled, as "Archive metadata block ... is missing!").
• rebuild_archives() / robust_item_ids(): self.repository.get(ptr_id) for an item pointers chunk, archive.py#L2710.
• rebuild_archives() / robust_iterator(): self.repository.get_many(items) for the item metadata stream, archive.py#L2735 (raises PackNotFound).
• rebuild_archives_directory() (--find-lost-archives): self.repository.get(chunk_id, read_data=False) for every object in the index, archive.py#L2527. This one crashes on any missing pack, even one holding only file content chunks.

verify_data() already handles this case (except Repository.PackNotFound: "pack ... is missing, N chunks are lost."), but the steps after it then crash on the same missing pack.

Also: with --archives-only and no --verify-data, a missing pack holding only file content chunks is not noticed at all (rc 0), because the file chunk check tests presence in the index, and the index still lists them. That is consistent with "the archives check trusts the index", but combined with the crashes above, it looks like the missing-pack case was only considered for the full check.

Possible fix

Instead of catching ObjectNotFound / PackNotFound at each read site, the archives check could do the same thing the repository check does before it uses a loaded index: list packs/, and drop the entries of packs that do not exist from self.chunks (remove_missing_pack_entries(), logging the missing packs as an error). Listing packs/ is cheap compared with the archives check. Then all the existing "chunk is missing" code paths report the lost objects correctly, the crashes are gone, and the rc-0 case above reports the lost file chunks, too.

Related: #9898 (vanished packs), #10069 (missing-pack detection in the repository check), #9998 (check tracking ticket). PR #10447 (--find-lost-archives reuses the --verify-data ids) happens to avoid the rebuild_archives_directory() crash when --verify-data is given, but rebuild_archives() still crashes afterwards.