fix(heartbeat): key the writer reservation off the per-issue tree (BLO-31443) by allyblockcast[bot] · Pull Request #1631 · Blockcast/paperclip

allyblockcast · 2026-09-03T11:53:38Z

Thinking Path

Paperclip is the open source app people use to manage AI agents for work

Agents execute in isolated runtimes, and the external_runtime_reservations single-writer index is what stops two runs writing one checkout

BLO-31282 (fix(heartbeat): keep the provisioned worktree under per-run isolation (BLO-31282) #1610) stopped overriding workspaceRoot for a run that already had a git worktree, which fixed base-checkout contamination but gave up an incidental exclusivity property

run:<runId> and workspace:<freshUuid> are unique per RUN, while the worktree they resolve to is keyed by ISSUE under the default per_issue runScope — so two concurrent runs of one issue held two distinct writer keys, both satisfied the index, and both wrote the same tree

This is not closable by configuration: an interaction wake is designed to run alongside a live run, and it never contends for issues.executionRunId at all

This pull request keys the writer reservation off the per-issue tree instead of the run, so same-issue runs collide on the existing index and the second is deferred

The benefit is that a checkout has one writer again, without giving back the sibling concurrency BLO-16842 added

Linked Issues or Issue Description

Fixes: BLO-31443 (Paperclip tracker) — key the isolation reservation off the resolved workspace path, not the run id
Refs BLO-31282 / fix(heartbeat): keep the provisioned worktree under per-run isolation (BLO-31282) #1610 — the change that traded this exclusivity away, deliberately and with sign-off
Refs BLO-16842 — sibling concurrency, whose invariant this PR narrows; preserved for different issues
Refs BLO-27858 / feat(workspace): warn a run when it shares a checkout with a live sibling (BLO-27858) #1393 — the closest neighbour, and complementary rather than duplicate: feat(workspace): warn a run when it shares a checkout with a live sibling (BLO-27858) #1393 detects the shared-checkout condition and warns; this PR prevents it. feat(workspace): warn a run when it shares a checkout with a live sibling (BLO-27858) #1393 is currently DIRTY against master. No conflict is expected (that PR adds a warning at run start; this one changes key derivation in resolveK8sRunIsolationIdentity), but whoever rebases feat(workspace): warn a run when it shares a checkout with a live sibling (BLO-27858) #1393 should read this diff first.
Searched open PRs for isolation|reservation|worktree|writer|workspace|31443|31403 — no competing implementation. BLO-31403 was a duplicate row of this one and was cancelled with no PR.

What Changed

resolveK8sRunIsolationIdentity takes a new optional perIssueWorkspaceTreeKey. When non-null it substitutes workspace-tree:<projectWorkspaceId>:<issueId> for a run-unique key, via the new withPerIssueWorkspaceTreeKey helper.
The call site in heartbeatService derives that key only when the run will work in a durable per-issue tree: not for statelessPrReview, and not for per_run runScope (both run-unique by construction).
New executionWorkspaceUsesGitWorktree predicate in execution-workspace-policy.ts. Deliberately mode-independent, because a real worktree and "workspace isolation was requested" are different questions — realizeExecutionWorkspace branches solely on workspaceStrategy.type === "git_worktree" and never consults the mode, so an agent- or issue-level strategy yields a worktree while the mode-derived flag reads false.
Three keys are deliberately left alone. The invariant is only ever replace a key that is run-unique, because a key already naming a shared tree is at least as strict as the per-issue key, so substituting it would loosen exclusivity:
- agent-shared:<agentId> is already stricter (one writer per agent);
- an explicitly reused workspace:<id> already names a tree that several issues may share — a per-issue key there would let two issues write it concurrently, i.e. the same defect in the opposite direction;
- statelessPrReview returns run-scoped isolation ahead of every other branch and must stay fully ephemeral.
isolationMode is untouched, so every filesystem root still derives from runId/persistedExecutionWorkspaceId exactly as before.
Tests: 233 lines in heartbeat-external-lifecycle-concurrency-flag.test.ts covering same-issue collision, the different-issue negative control, each left-alone key, and the session-resume coupling below.

Verification

npx vitest run server/src/__tests__/heartbeat-external-lifecycle-concurrency-flag.test.ts
  → 28 passed (28)
npx vitest run server/src/__tests__/heartbeat-external-runtime-retry.test.ts \
              server/src/__tests__/heartbeat-workspace-session.test.ts
  → 255 passed (255)
npm run typecheck  → exit 0

Mapping to the issue's acceptance criteria:

AC	how it is met
same-issue runs cannot both hold a writer reservation	both resolve to one `workspace-tree:` key → collide on `external_runtime_reservations_active_isolation_writer_idx` → second deferred. Asserted directly.
different issues still parallel (BLO-16842)	key embeds `issueId`, so keys differ. Asserted as the negative control.
`statelessPrReview` unaffected	provable by construction, not just by test — that branch returns before the helper is reachable, and the call site also gates on `paperclipPrReview === null`.
holds at effective concurrency 1 and above	at concurrency 1 the `run:` branch is unreachable; both runs already share `agent-shared:<agentId>` and serialize. The issue's own argument for this AC does not hold — `effectiveMaxConcurrentRuns` is a pure function of configured policy, not of live counts, so the BLO-12990 staleness floor frees a scheduling slot but cannot change key derivation. Recorded on the issue; the AC is satisfied, its stated mechanism was wrong.

Risks

One behavioural change beyond the stated scope, called out because it is easy to miss. isolationKey is not read only by the reservation — it is also stamped onto persisted session params and gates saved-session resume through sessionParamsMatchIsolation (consumed at the isolation_mismatch requeue). Making the key stable across runs of one issue therefore flips that guard from reject to accept, where a run:<runId> key could never match a previous run's stamp.

That is intended, and it is the pairing BLO-31282 already implies: such a run works in a durable per-issue worktree, so the tree its session refers to really is the same one next run. It is pinned by a test with its own negative control, so if a future change wants the reservation key to move without moving session scope, the two need separating and that assertion is where it shows up. Reviewers: this is the assumption most worth challenging.

Otherwise low risk, and conservative in the one case where issue and path disagree: an issue retitled between runs resolves to a new directory while keeping its id, so this over-serializes. Serializing two runs that could have been parallel costs latency; letting two runs share one tree corrupts a checkout.

No migration. No API change. tmpRoot's hash input and sessionScope.isolationKey move with the key; no other root does.

Model Used

Claude Opus 4.8 (claude-opus-5[1m], 1M context), extended thinking, with tool use and code execution via Claude Code.

Checklist

…O-31443) BLO-31282 (#1610) stopped overriding `workspaceRoot` for a run that already had a git worktree, which fixed base-checkout contamination but gave up an incidental exclusivity property. `run:<runId>` and `workspace:<freshUuid>` are both unique per RUN, while the worktree they resolve to is keyed by ISSUE under the default `per_issue` runScope. Two concurrent runs of one issue therefore held two distinct writer keys, both satisfied `external_runtime_reservations_active_isolation_writer_idx`, and both wrote the same tree. `resolveK8sRunIsolationIdentity` now takes `perIssueWorkspaceTreeKey` and substitutes `workspace-tree:<projectWorkspaceId>:<issueId>` for run-unique keys only, so same-issue runs collide on that index and the second is deferred. `isolationMode` is untouched, so every filesystem root still derives from `runId`/`persistedExecutionWorkspaceId` exactly as before. The invariant: only ever replace a key that is RUN-UNIQUE. Three keys are left alone, each for its own reason -- `agent-shared:<agentId>` is already stricter than per-tree, an explicitly reused `workspace:<id>` already names a tree that several issues may share, and stateless PR review must stay fully ephemeral. Substituting any of them would loosen exclusivity rather than tighten it. Co-Authored-By: Claude <noreply@anthropic.com>

…BLO-31443) `isolationKey` is not read only by the writer reservation: it is also stamped onto persisted session params and gates saved-session resume via `sessionParamsMatchIsolation`, consumed at the `isolation_mismatch` requeue. Making the key stable across runs of one issue therefore FLIPS that guard from reject to accept, where a `run:<runId>` key could never match a previous run's stamp. That is intended rather than incidental, and it is the pairing BLO-31282 already implies: such a run works in a durable per-issue worktree, so the tree its session refers to really is the same one next run. Pinned with its own negative control so the coupling stays a decision -- if a future change wants the reservation key to move without moving session scope, the two need separating and this assertion is where that shows up. Co-Authored-By: Claude <noreply@anthropic.com>

allyblockcast · 2026-09-03T11:53:43Z

🔗 Paperclip issue: BLO-12990
🔗 Paperclip issue: BLO-31282
🔗 Paperclip issue: BLO-27858
🔗 Paperclip issue: BLO-31443
🔗 Paperclip issue: BLO-31403
🔗 Paperclip issue: BLO-16842

allyblockcast · 2026-09-03T11:53:43Z

🔗 Paperclip issue: BLO-12990
🔗 Paperclip issue: BLO-31282
🔗 Paperclip issue: BLO-27858
🔗 Paperclip issue: BLO-31443
🔗 Paperclip issue: BLO-31403
🔗 Paperclip issue: BLO-16842

github-actions · 2026-09-03T13:27:14Z

@ally head 2de08ee has been awaiting review for 1.6h with no review on either surface (pulls/1631/reviews carries no consolidated report for this head, no ## Ally comment either) -- automated sweep (BLO-22892 / BLO-28203), not a human/agent re-ask.

Requested a review from @allyblockcast directly (native GitHub review request, not just this comment) against current head 2de08ee.

github-actions · 2026-09-03T16:37:03Z

@ally head 2de08ee has been awaiting review for 4.7h with no review on either surface (pulls/1631/reviews carries no consolidated report for this head, no ## Ally comment either) -- automated sweep (BLO-22892 / BLO-28203), not a human/agent re-ask.

Requested a review from @allyblockcast directly (native GitHub review request, not just this comment) against current head 2de08ee.

github-actions · 2026-09-03T19:17:00Z

@ally head 2de08ee has been awaiting review for 7.4h with no review on either surface (pulls/1631/reviews carries no consolidated report for this head, no ## Ally comment either) -- automated sweep (BLO-22892 / BLO-28203), not a human/agent re-ask.

Requested a review from @allyblockcast directly (native GitHub review request, not just this comment) against current head 2de08ee.

github-actions · 2026-09-03T21:20:29Z

@ally head 2de08ee has been awaiting review for 9.4h with no review on either surface (pulls/1631/reviews carries no consolidated report for this head, no ## Ally comment either) -- automated sweep (BLO-22892 / BLO-28203), not a human/agent re-ask.

Requested a review from @allyblockcast directly (native GitHub review request, not just this comment) against current head 2de08ee.

CTO and others added 2 commits September 3, 2026 10:58

github-actions Bot requested a review from allyblockcast September 3, 2026 13:27

github-actions Bot requested review from allyblockcast and removed request for allyblockcast September 3, 2026 16:37

github-actions Bot requested review from allyblockcast and removed request for allyblockcast September 3, 2026 19:16

github-actions Bot requested review from allyblockcast and removed request for allyblockcast September 3, 2026 21:20

Navigation Menu

Uh oh!

fix(heartbeat): key the writer reservation off the per-issue tree (BLO-31443) - #1631

fix(heartbeat): key the writer reservation off the per-issue tree (BLO-31443)#1631
allyblockcast[bot] wants to merge 2 commits into
masterfrom
BLO-31443-key-the-isolation-reservation-off-the-resolved-workspace-path-not-the-run-id-restores-same-issue-workspace-exc

allyblockcast Bot commented Sep 3, 2026

Uh oh!

allyblockcast Bot commented Sep 3, 2026

Uh oh!

allyblockcast Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants

Uh oh!

Conversation

allyblockcast Bot commented Sep 3, 2026

Thinking Path

Linked Issues or Issue Description

What Changed

Verification

Risks

Model Used

Checklist

Uh oh!

allyblockcast Bot commented Sep 3, 2026

Uh oh!

allyblockcast Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

github-actions Bot commented Sep 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

0 participants