Skip to content

compact: reclaim superseded duplicates left after a full check --repair #10471

Closed

Description

@mr-raj12

Follow-up to #10443 (review there).

When a crashed borg create is re-run and the archive is kept, the second run stores the lost chunks again, so the crashed run's packs hold only copies of chunks indexed elsewhere (superseded duplicates: unindexed objects whose chunk id the index maps to another location).

  • A full borg check --repair rebuilds the index with one entry per chunk id (build_chunkindex_from_repo), so these copies stay unindexed.
  • compact_packs rewrites a pack only if indexed - used is above the threshold. For these packs it is 0, so they are never rewritten and compact_pack never drops the duplicates.
  • Every borg compact logs the unindexed-bytes hint again, and the space is never reclaimed.

Repro: kill -9 a borg create of a 1.5 GB random file after 3s, re-run the same create, keep the archive. 1.07 GB stays unindexed and the packs stay at 2.5G through two full borg check --repair + borg compact runs. borg check is rc 0.

Proposal: classify unindexed objects in compact_packs pass 1 as well (as compact_pack does via superseded_gap_ranges), so superseded bytes count toward the rewrite threshold and a pack holding only superseded duplicates can be deleted. See the TODO(#9868 follow-up) in compact_pack and the TODO: in compact_packs.

Activity

  1. added 6 commits that reference this issue on Oct 3, 2026
    9121c9a
    a353355
    9f5ac68
    f3b98cf
    86efd29
    8211bb7
  2. added a commit that references this issue on Oct 5, 2026
    400185b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions