sync: snapshot the index on every successful sync and ride a copy to the destination

[mbertschler]

What

Implements #75. The catalog should be as redundant as the data it describes, so after every successful sync squirrel now:

1. Snapshot-on-sync (local). Takes one VACUUM INTO snapshot of the global index (reusing store.Backup) to a local tier (default <dir of db>/backups/), named index-<ISO8601>-run-<id>.db, with keep-N rotation.
2. Cloud ride-along (destination syncs only). Rides the same snapshot file along to <dest.root>/<volume>/.squirrel-index/index-<ISO8601>-run-<id>.db via the rclone wrapper (copyto), then rotates that dir to cloud_keep (rclone lsf newest-N + deletefile).

One VACUUM per squirrel sync invocation, fanned out to the local tier and every destination — never one VACUUM per pair.

Decisions honoured

• Local folder indexing + querying #1 Payload is the full global index.db (no per-volume filtering); a copy lands under each synced volume's .squirrel-index/.
• Local folder indexing + querying (issue #1) #2 On by default, zero-config: an absent [backups] table = enabled with defaults.
• Introduce 'volumes' concept: schema v2 migration, implicit creation, vocab rename #3 Filename index-<ISO8601>-run-<id>.db — lexically sortable and traceable to its producing run.

Config

[backups]
enabled    = true   # local snapshot-on-sync (default true)
dir        = ""     # default: <dir of db>/backups
keep       = 7      # local rotation
cloud      = true   # ride-along to destination buckets (default true)
cloud_keep = 7      # snapshots kept per <dest>/<volume>/.squirrel-index/

enabled = false disables both halves; cloud = false keeps the local snapshot but uploads nothing.

Guards (non-negotiable)

• .squirrel-index/ added to all three reserved-path filters: the --filter lists in buildRcloneArgs and buildRestoreArgs, plus isReservedSyncPath/isReservedFolderPath in peer-sync.
• A snapshot/upload failure surfaces on Report.SnapshotErr (mirroring FinishErr) and is logged — it never flips a successful sync to failed or mutates rep.Status.
• Snapshot is taken after the run's terminal row is committed (SyncNode was split into runNodeSession so its deferred finishRun fires first).
• No peer ride-along; restore runs take no snapshot. Scheduled syncs on the agent (each node) snapshot too.

Tests

• Cloud ride-along: snapshot present under .squirrel-index/, is the global catalog (carries rows for a volume never synced to that destination), opens cleanly + passes integrity check, local-tier copy present with matching name, filename carries the run id.
• .squirrel-index/ excluded from sync (source-leak test), restore + sync arg builders (filter assertion), and peer-sync (isReservedSyncPath/isReservedFolderPath).
• Injected backup error → run status unchanged, surfaced on SnapshotErr.
• enabled=false (nil Snapshotter → no-op) and cloud=false (local snapshot, no upload).
• Local + cloud rotation bound to N.
• Config: defaults when absent, overrides, enabled=false/cloud=false, negative-keep rejection, unknown-field rejection.

go vet ./..., go test ./..., and go test -race ./sync/... all pass.

Out of scope (per ticket)

Litestream; peer ride-along; snapshot on restore; encrypting the snapshot at rest (docs note recommends a private bucket / SSE — added to the README).

Closes #75

---

Generated by Claude Code

[Copilot]

Pull request overview

This PR implements issue #75 by adding automatic SQLite index snapshots after successful sync runs, storing them locally with rotation and (for destination syncs) uploading the same snapshot as a “ride-along” into each destination volume’s .squirrel-index/, also with rotation. It also ensures .squirrel-index/ is treated as a reserved path and excluded from sync/restore transfers and peer-sync discovery.

Changes:

• Add Snapshotter to take one VACUUM INTO snapshot per squirrel sync invocation and fan it out to local + destination copies, surfacing failures via Report.SnapshotErr without failing the sync.
• Exclude .squirrel-index/ in rclone sync/restore filters and in peer-sync reserved-path checks.
• Add [backups] config block with defaults (enabled by default) and tests; wire snapshot behavior into CLI sync and agent scheduled syncs.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
sync/sync.go	Adds .squirrel-index constant, Options.Snapshot, Report.SnapshotErr, and filter exclusions for sync/restore.
sync/snapshot.go	Implements Snapshotter (local snapshot, cloud ride-along upload, local/cloud rotation).
sync/snapshot_test.go	Adds tests covering ride-along, disable flags, error surfacing without status flip, rotation bounds, and reserved-path guards.
sync/rclone.go	Adds “plain” rclone helpers (copyto, lsf, deletefile) to support snapshot ride-along and rotation.
sync/node.go	Ensures snapshot hook runs after peer-sync terminal run state is committed; expands reserved-path predicates.
README.md	Documents snapshot-on-sync behavior and destination layout including .squirrel-index/.
config/config.go	Adds resolved Backups config to Config and resolves it during load.
config/backups.go	Introduces [backups] config with defaults and validation.
config/backups_test.go	Tests defaults, overrides, disabling, negative retention rejection, and unknown-field rejection.
cmd/squirrel/sync.go	Wires Snapshotter creation into squirrel sync (shared across pairs).
cmd/squirrel/agent.go	Enables snapshot-on-sync for scheduled sync kicks via agent.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.