Skip to content

Checkpoints V2: add migration option#891

Merged
Soph merged 12 commits intomainfrom
add-migrate-v2-command-rebased
Apr 9, 2026
Merged

Checkpoints V2: add migration option#891
Soph merged 12 commits intomainfrom
add-migrate-v2-command-rebased

Conversation

@Soph
Copy link
Copy Markdown
Collaborator

@Soph Soph commented Apr 9, 2026

this is a recreation of #855 due to #869


Note

Medium Risk
Adds a new migration workflow that rewrites checkpoint data across git refs (including tree surgery and transcript compaction/backfill), which could impact stored checkpoint integrity if edge cases slip through despite tests.

Overview
Adds a hidden entire migrate --checkpoints v2 command that migrates committed checkpoints from v1 storage to v2 (/main + /full/current), iterating checkpoints/sessions and optionally generating transcript.jsonl via compaction.

The migration is idempotent and includes repair/backfill paths: it detects already-migrated v2 checkpoints, restores missing /full/current raw transcript artifacts, backfills missing compact transcripts, and copies task metadata trees into v2.

Prompt serialization is centralized via new checkpoint.PromptSeparator, JoinPrompts, and SplitPromptContent helpers (with tests), and v1/v2 prompt writing is updated to use the canonical join format.

Reviewed by Cursor Bugbot for commit cef06e7. Configure here.

Entire-Checkpoint: 209a37190167
Entire-Checkpoint: c9595c52ab4a
Entire-Checkpoint: 9f07aeebbf93
Entire-Checkpoint: f1c37c8efc47
Entire-Checkpoint: 36db97269a69
Entire-Checkpoint: 93066e1dac3c
Entire-Checkpoint: 4fdb72622b7f
Entire-Checkpoint: 730e93f6b572
Copilot AI review requested due to automatic review settings April 9, 2026 12:00
@Soph Soph requested a review from a team as a code owner April 9, 2026 12:00
@Soph Soph enabled auto-merge April 9, 2026 12:05
@Soph Soph merged commit 96d55ef into main Apr 9, 2026
8 checks passed
@Soph Soph deleted the add-migrate-v2-command-rebased branch April 9, 2026 12:06
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a hidden entire migrate command to migrate committed checkpoints from the v1 branch layout to the v2 ref layout, with support for backfilling missing compact transcripts and repairing incomplete v2 /full/current artifacts. Also centralizes prompt serialization via shared helpers to keep v1/v2 prompt writing consistent.

Changes:

  • Register hidden migrate command (--checkpoints v2) and implement v1→v2 migration + repair/backfill logic.
  • Add comprehensive unit tests for migration scenarios (idempotency, multi-session, backfill, repair, task metadata splice).
  • Introduce PromptSeparator/JoinPrompts/SplitPromptContent and switch v1/v2 checkpoint writers to use JoinPrompts.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
cmd/entire/cli/root.go Registers the new hidden migrate command on the root CLI.
cmd/entire/cli/migrate.go Implements v1→v2 checkpoint migration, repair, transcript compaction backfill, and task tree splicing.
cmd/entire/cli/migrate_test.go Adds unit tests covering core migration + repair/backfill behaviors.
cmd/entire/cli/checkpoint/v2_committed.go Uses shared JoinPrompts when writing prompts in v2.
cmd/entire/cli/checkpoint/prompts.go Adds canonical prompt separator and join/split helpers.
cmd/entire/cli/checkpoint/prompts_test.go Tests prompt join/split round-trip behavior.
cmd/entire/cli/checkpoint/committed.go Uses shared JoinPrompts when writing prompts in v1.

Comment on lines +154 to +158
// Already in v2 — check if any aspect of sessions are missing and backfill
if existing != nil {
repaired, repairErr := repairPartialV2Checkpoint(ctx, repo, v1Store, v2Store, info, existing)
if repairErr != nil {
return repairErr
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When a checkpoint already exists in v2, migration only repairs /full/current transcript artifacts and backfills compact transcripts; it never attempts to copy task metadata trees from v1 into v2. If the initial migration skipped task metadata (e.g., copyTaskMetadataToV2 failed and only logged a warning), rerunning entire migrate --checkpoints v2 will never backfill those task files, leaving task checkpoints permanently incomplete in v2. Consider adding a best-effort task-metadata backfill in the existing != nil path (e.g., detect v1 task metadata and splice it into /full/current if missing).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit cef06e7. Configure here.

}

return repaired, nil
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-run silently skips sessions missing from v2

Medium Severity

When the initial migration of a multi-session checkpoint is interrupted (e.g., session 0 writes to v2 but session 1 fails), re-running the migration takes the existing != nil path. Both repairPartialV2Checkpoint and backfillCompactTranscripts only iterate over len(v2Summary.Sessions) — the sessions already in v2. Sessions present in v1 but missing from v2 are never added. The checkpoint is reported as "skipped (already in v2)" despite being incomplete, making the migration appear successful when sessions are silently lost from v2.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit cef06e7. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants