Skip to content

fix(T13891): one project snapshot per run for the unified cleo.db - #2364

Open
kryptobaseddev wants to merge 2 commits into
mainfrom
fix/T13891-one-project-snapshot
Open

kryptobaseddev wants to merge 2 commits into
mainfrom
fix/T13891-one-project-snapshot

Conversation

@kryptobaseddev

Copy link
Copy Markdown
Owner

T13891: session-end backups of the unified project store.

What the cleocode store actually shows

This is a read-only listing of .cleo/backups/sqlite/ (25 GB, 84 files), using inodes and link counts. It does not match the task's "three full copies per Stop" premise:

  • Hard links, not copies. On main, vacuumIntoBackupAll already VACUUMs cleo.db once per run. It then hard-links that file under brain-* and conduit-*; brain-20260930-135427.db and conduit-20260930-135428.db share an inode, with link count 2.
  • Stale links kept alive. Each prefix rotates on its own schedule (selectSnapshotsToKeep). tasks-* also takes the per-write routine checkpoints, so it rotated out its half of each link. Four old brain-/conduit- pairs (Sep 30 ×3, Oct 3; about 1.3 GB each, 5.2 GB in total) are still alive only through those names. They will never be pruned, because nothing writes those prefixes now.
  • Full copies elsewhere. On a filesystem without hard links (exFAT, some network mounts), the fallback copied the file. That really is three full copies per run.
  • The biggest consumer is not session end. It is cleo backup add (createBackup, type snapshot): ten cleo.db.snapshot-* copies of about 1.35 GB each, roughly 13 GB in total. Beyond that, five tasks-*.db (6.7 GB) and two cleo.db.migration-* (2.5 GB).
  • No session-end entries since Oct 3. .cleo/logs/session-end-snapshot.log has none after then, so the recent tasks-*.db files come from vacuumIntoBackup checkpoints. Worth a separate look.

Fix

snapshotOne (packages/core/src/store/sqlite-backup.ts): when the physical file was already snapshotted earlier in the run, the target now returns 'linked' (covered) and writes nothing. It no longer creates a hard link or copy. Every run, session end included, therefore leaves exactly one tasks-*.db for the unified store. The duplicate check now happens before a filename is chosen, so a covered prefix never waits for a free name either. The hard-link helpers are removed.

  • Rotation (AC2): tasks-* keeps its existing time-spread retention, unchanged.
  • Readers (AC3):
    • cleo restore backup --snapshot already takes tasks-*.db, unchanged.
    • listBrainBackups now returns the project snapshots (tasks-*) plus older brain-* files, newest first.
    • backup verify counts tasks-*.db for the brain and conduit roles, as it already counts cleo.db.<backupId>.
    • listSqliteBackupsAll still lists the legacy brain-*/conduit-* buckets.
  • The gate result keeps its shape: snapshotted: ['tasks','brain','conduit'], linked: ['brain','conduit'].

Tests (AC4)

In sqlite-backup.test.ts:

  • "one project snapshot per run": one VACUUM, one tasks-* file, and no brain-*/conduit-*.
  • The no-hard-link (ENOTSUP) case now asserts one file and no copy, and that listBrainBackups returns that file.
  • The listing test expects listBrainBackups to include tasks-*.

sqlite-backup-real-process.test.ts (T10316) now asserts one tasks-* and zero brain-*/conduit-* in a real process.

These test changes are deliberate, because the snapshot layout changed.

No local heavy runs (owner directive). CI is the test evidence. Biome on the changed files and gates 28 and 32 (--base origin/main) pass locally.

Not in scope

  • Existing duplicates aren't deleted. The legacy brain-*/conduit-* files already on disk stay, so restore can still list them. Reclaiming the four orphaned pairs (5.2 GB here) needs a deliberate cleanup step, for example a doctor job.
  • createBackup retention: ten full cleo.db copies from cleo backup add is the larger disk cost.

🤖 Generated with Claude Code

kryptobaseddev and others added 2 commits October 17, 2026 19:17
tasks, brain and conduit share the project cleo.db, and
vacuumIntoBackupAll VACUUMed it once but then hard-linked the file under
brain-* and conduit-* (a full copy where hard links are unsupported).
Each prefix rotates on its own schedule, so the links diverged: on the
cleocode store four old brain/conduit pairs (5.2 GB) outlived their
tasks-* twins and were never pruned. Now the first target on the file
writes the one tasks-*.db snapshot and the others report 'linked'
(covered) without writing a file. listBrainBackups and backup verify
treat the project snapshot as holding brain and conduit; older
brain-*/conduit-* files stay listed. Tests assert one project snapshot
per run, including where hard links are unsupported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shot

After T13891 a run writes one tasks-*.db for the unified store, but three
recovery readers matched only role-prefixed files, so `cleo backup
recover brain|conduit` would pick the last pre-upgrade brain-* copy (a
whole cleo.db, rolling the store back) or fail with E_NO_SNAPSHOT on a
fresh install. vacuumSnapshotPrefixes() names tasks as a prefix that
holds brain and conduit; buildVacuumSnapshotRegex (and with it
collectSnapshotCandidatesForRole / recoverProjectStore), the
snapshotMatchesPin vacuum branch and recover-brain-db's
VACUUM_FILENAME_RE now pick the freshest file across both prefixes.
Tests: brain and conduit recover from a newer tasks-* over an older
role-prefixed file; recover-brain-db does the same; the real-process
test asserts recovery finds the one project snapshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant