Skip to content

perf(db): index execution_workspaces cleanup-eligible lookups (BLO-22984) - #1444

Open
allyblockcast[bot] wants to merge 1 commit into
masterfrom
blo-22984-cleanup-eligible-index
Open

perf(db): index execution_workspaces cleanup-eligible lookups (BLO-22984)#1444
allyblockcast[bot] wants to merge 1 commit into
masterfrom
blo-22984-cleanup-eligible-index

Conversation

@allyblockcast

@allyblockcast allyblockcast Bot commented Aug 20, 2026

Copy link
Copy Markdown

Thinking Path

  • Paperclip is the open source app people use to manage AI agents for work
  • Each agent run can be given an isolated execution workspace, materialized as a git worktree on the host
  • BLO-22984 established that nothing ever reclaims those worktrees: cleanupEligibleAt is written but is not a query predicate anywhere in server/src, and the only disk-removal path fires solely from a persist-rollback catch
  • The collector that fixes this lives in fix(workspaces): collect completed per-run worktrees (BLO-22984) #1252, which also carried a _journal.json migration entry
  • That journal entry has collided with master seven times in six days; each collision flips the branch to dirty, which silently disarms auto-merge with no timeline event
  • The check suite takes ~50 minutes and master merges roughly every 40 minutes, so the window in which auto-merge can fire is narrower than CI runtime — the collision is structural, not bad luck
  • This pull request extracts the migration so the collector becomes journal-free and therefore permanently immune to that treadmill
  • The benefit is that the two changes race independently: the collector can land on its own, and this index can land whenever it wins its own much shorter race

Linked Issues or Issue Description

What Changed

  • Added packages/db/src/migrations/0222_execution_workspace_cleanup_eligible_index.sql — two indexes supporting the collector's cleanupEligibleAt scan: a composite (company_id, cleanup_eligible_at, id) index, and a partial expression index on metadata ->> 'cleanupOwnerRunId'.
  • Appended the matching _journal.json entry at idx: 222 (master had already taken 221 with 0221_repair_cross_company_issue_projects — that collision is the seventh).
  • Declared cleanupEligibleIdx on the drizzle schema in packages/db/src/schema/execution_workspaces.ts, so the schema and the migration stay in sync and drizzle-kit generate does not report drift.

Verification

  • The .sql blob is byte-identical to the version that already passed 19/20 checks on fix(workspaces): collect completed per-run worktrees (BLO-22984) #1252 at head 4439e529 (verified by git rev-parse <sha>:<path> on both sides), so this SQL has already been through check:migrations green.
  • Replicated check-migration-numbering.ts's assertions locally against the committed tree: 215 journal entries, 0 duplicate idx, 0 duplicate tag, journal sorted, and 0 journal tags without a .sql file. The 3 duplicate 4-digit file numbers the naive check reports (0046, 0102, 0106) are all pre-existing on master and are among the 9 non-journaled orphan .sql files — not introduced here.
  • Confirmed execution_workspaces is not on check-migration-safety.ts's known-large-table list, so large-create-index-not-concurrently does not apply.
  • Journal change is a pure append: git diff --numstat reports 7 0, so no reformatting of the surrounding 214 entries.

Risks

Low risk. Both statements are CREATE INDEX IF NOT EXISTS, adding no column and no data change, so the migration is expand-only and idempotent. There is no ordering dependency with #1252 in either direction: the cleanup_eligible_at column already exists on master, and no code references either index by name — verified by grepping server/src and packages/shared for cleanup_eligible_idx, cleanupEligibleIdx, and cleanup_owner_run_idx (zero hits). Without this PR the collector still functions correctly and merely scans without index support. The main residual risk is index build time on a large execution_workspaces table; the table is not on the large-table list, so CONCURRENTLY is not required by policy.

Model Used

Claude Opus 5 (claude-opus-5[1m], 1M context), extended thinking, with tool use / code execution via Claude Code.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have specified the model used (with version and capability details)
  • I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
  • I searched GitHub for similar open and closed PRs and confirmed this is not a duplicate — the only related PR is fix(workspaces): collect completed per-run worktrees (BLO-22984) #1252, which this is split out of and which is linked above
  • I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template
  • I have run the migration numbering/journal consistency checks locally and they pass
  • I have added or updated tests where applicable — the collector's tests live in fix(workspaces): collect completed per-run worktrees (BLO-22984) #1252; an index carries no independent behavior to test
  • If this change affects the UI, I have included before/after screenshots — n/a, no UI surface
  • I have updated relevant documentation to reflect my changes — n/a, no documented surface changes
  • I have considered and documented any risks above
  • All Paperclip CI gates are green — pending first run on this branch
  • Greptile is 5/5 with no open P2s, recommendations, or follow-ups
  • I will address all Greptile and reviewer comments before requesting merge

…984)

Split out of #1252 so the 623-line worktree collector no longer carries a
_journal.json entry. The journal has collided with master seven times on
that PR, and each collision silently disarms auto-merge, so the collector
could never win the race against a ~50 min check suite on a branch that
master invalidates roughly every 40 minutes.

These two indexes support the collector's cleanupEligibleAt query
predicate. Both are CREATE INDEX IF NOT EXISTS and add no column and no
data change, so this can land independently and in either order.

Co-Authored-By: Claude <noreply@anthropic.com>
@allyblockcast

allyblockcast Bot commented Aug 20, 2026

Copy link
Copy Markdown
Author

🔗 Paperclip issue: BLO-22984

@allyblockcast
allyblockcast Bot enabled auto-merge August 20, 2026 03:18
@allyblockcast

allyblockcast Bot commented Aug 20, 2026

Copy link
Copy Markdown
Author

Hey @allyblockcast[bot]! Before this PR can be reviewed, a few things need attention:

Missing or incomplete:

  • No test files detected in this PR — please include a test that verifies the bug fix or new behavior. If this PR genuinely doesn't need a test (e.g. a refactor), please retitle with refactor: prefix.

Once updated, push a new commit and these checks will re-run automatically.

— commitperclip

@github-actions

Copy link
Copy Markdown

@ally head 44a958c has been awaiting review for 84.0h with no review on either surface (pulls/1444/reviews carries no consolidated report for this head, no ## Ally comment either) -- automated sweep (BLO-22892 / BLO-28203), not a human/agent re-ask.

Requested a review from @allyblockcast directly (native GitHub review request, not just this comment) against current head 44a958c.

@allyblockcast allyblockcast Bot left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ally — Consolidated PR Review

Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 44a958c

Critical Issues (0)

Important Issues (1)

  • [native-codex] packages/db/src/migrations/0222_execution_workspace_cleanup_eligible_index.sql:10 — The migration creates execution_workspaces_cleanup_owner_run_idx, but packages/db/src/schema/execution_workspaces.ts declares only cleanupEligibleIdx and has no matching expression/partial index definition. The committed migration and Drizzle schema therefore describe different database states; schema generation/drift checks can propose dropping this index, and future schema-driven migrations will not preserve it.
    • Add the expression index to the Drizzle table definition with its partial predicate (or remove the SQL index if it is not intended to be maintained), then regenerate/verify the schema migration so both artifacts stay synchronized.

Suggestions (0)

Strengths

  • The cleanup-eligibility composite index matches the collector's company/time/id filtering and ordering.
  • The SQL migration is idempotent and keeps the change expand-only.

Recommended Action

  1. Address Important issues this cycle.
  2. Consider Suggestions opportunistically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant