Skip to content

fix(issues): fence a second run off a live-locked pending in_review stage (BLO-22666) - #1366

Merged
kkroo merged 1 commit into
masterfrom
cto/blo-22666-inreview-ownership
Aug 16, 2026
Merged

fix(issues): fence a second run off a live-locked pending in_review stage (BLO-22666)#1366
kkroo merged 1 commit into
masterfrom
cto/blo-22666-inreview-ownership

Conversation

@allyblockcast

@allyblockcast allyblockcast Bot commented Aug 15, 2026

Copy link
Copy Markdown

Thinking Path

  • Paperclip is the open source app people use to manage AI agents for work
  • Agents claim work through a run-scoped execution lock (checkoutRunId / executionRunId); that lock is what stops two runs doing the same job twice
  • BLO-18858 found the lock being ignored: two concurrent runs of the SAME agent duplicated identical work
  • fix(agents): tell the truth about inbox-lite status filter (BLO-18858) #1117 fixed the checkout half — a pending review stage now stays in_review and is atomically checkoutable — but it never touched server/src/routes/issues.ts, so nothing asserts run-ownership once the row is in_review
  • Route authorization early-returns on status !== "in_progress", so the same agent's second run can still PATCH the issue to done, or approve by comment, on top of the stage run A is holding
  • This pull request fences that one case, and gives a drifted currentParticipant an atomic way to claim the stage it is already authorized to decide
  • The benefit is that a pending review stage has exactly one owner, without taking approval rights away from the reviewers who are supposed to have them

Linked Issues or Issue Description

What Changed

  • isForeignRunOfLockedPendingReview (routes/issues.ts) — predicate identifying the one actor that must be fenced: a second run of the issue's own assignee, on a live-locked, pending in_review stage.
  • assertPendingReviewRunOwnership (services/issues.ts) — the in_review counterpart of assertCheckoutOwner, kept as a separate entry point rather than widening that one. It clears locks held by terminal runs first (otherwise a dead run A would fence its own agent's run B forever), re-reads, and only then fences on a live foreign run.
  • Three fallthroughs closedensureAgentCheckoutOwnership (PATCH), the approval-shaped-comment branch, and the execution-stage decision path.
  • AC3 participant claim — a pending stage pinned to a currentParticipant whose id has drifted off assigneeAgentId can now take the lock. The claim writes only the lock columns; assigneeAgentId, assigneeUserId, status and startedAt are untouched, so it cannot become a back door to general issue ownership. The participant pin is re-asserted inside the UPDATE as a compare-and-swap.
  • 11 tests added to issue-agent-mutation-ownership-routes.test.ts, four of them negative controls.

Verification

npx tsc --noEmit -p server/tsconfig.json                                   # clean
npx vitest run src/__tests__/issue-agent-mutation-ownership-routes.test.ts # 221 passed
npx vitest run src/__tests__/issue-stale-execution-lock-routes.test.ts     # 53 passed

Each guard was verified load-bearing — neutered individually, then re-run:

reverted result
isForeignRunOfLockedPendingReviewfalse 3 fail: the PATCH fence, its 409 surfacing, the approval-comment fence
allowExecutionStageParticipantClaimfalse 1 fails: the drifted-participant claim

The four negative controls stayed green under both reverts, which is the point — they must be insensitive to the fence:

  • a non-assignee peer reviewer holding no checkout still approves
  • an unlocked pending in_review row is still freely checkoutable (does not re-close the door fix(agents): tell the truth about inbox-lite status filter (BLO-18858) #1117 opened)
  • an ordinary (non-approval) comment from run B still lands, so a losing run can leave its findings behind
  • the run that actually holds the lock is never fenced

One flake seen and run to ground: issue-stale-execution-lock-routes.test.ts failed once in afterEach with an FK violation on agent_wakeup_requests — an async wakeup insert racing teardown. Re-run 3× on this branch: 53/53 each time. It is not caused by this change; the affected test's fixture has assigneeAgentId !== actor, so the new fence provably does not arm on it.

Risks

The regression to fear is over-fencing, not under-fencing. By the time control reaches this predicate the actor has already cleared the authorization boundary, so it may legitimately be a mention-granted peer reviewer, a manager-chain actor, a recovery owner, a human, or a drifted currentParticipant — none of whom hold the checkout, and all of whom are supposed to be able to approve. The assigneeAgentId === actor term is what keeps them out of the fence and must not be widened; doing so would deadlock every review stage. Four prior Criticals on this predicate came from exactly that over-reach, which is why four of the eleven tests exist only to pin the non-fenced paths.

Placement in ensureAgentCheckoutOwnership is deliberate: after the recovery-owner and unassigned early-returns so no rescue path can be fenced, and before the blocked-correction and participant-decision returns, which are the two ways run B would otherwise slip past.

Not rebased from #1212 or #1120. Both predate #1117 and are built on the withPendingInReviewRunOwnershipGuard substrate that master replaced; I tested it rather than assuming — git rebase origin/master on #1212's head conflicts in routes/issues.ts and services/issues.ts, the exact two files #1117 rewrote, and #1212 is 225 commits behind with its checkout half now redundant and an out-of-scope external-objects.ts change. No migration, no API-shape change.

Model Used

  • Claude Opus 5 (claude-opus-5[1m], 1M context), extended thinking, via Claude Code with tool use.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have specified the model used (with version and capability details)
  • I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
  • I have searched GitHub for duplicate or related PRs and linked them above
  • I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template
  • I have run tests locally and they pass
  • I have added or updated tests where applicable
  • If this change affects the UI, I have included before/after screenshots — n/a, server-side only
  • I have updated relevant documentation to reflect my changes — behaviour is documented in-code; the user-facing paperclipCheckoutIssue description already shipped with fix(agents): tell the truth about inbox-lite status filter (BLO-18858) #1117
  • I have considered and documented any risks above
  • All Paperclip CI gates are green — pending first CI run
  • Greptile is 5/5 with no open P2s, recommendations, or follow-ups — pending review
  • I will address all Greptile and reviewer comments before requesting merge

Note on the backlink comments below: both auto-posted backlinks point at BLO-18858, not this PR's issue. That is an artifact of the execution workspace (bfc31d3b, named BLO-18858) — the backlink bot follows the workspace's sourceIssueId rather than the issue being worked. The correct Paperclip issue for this PR is BLO-22666, as stated above.

…tage (BLO-22666)

AC2: run-ownership was never asserted on an `in_review` row. ensureAgentCheckoutOwnership
early-returns on `status !== "in_progress"`, and the approval-shaped-comment branch gates
on identity only, so the SAME agent's second run could decide the stage run A was sitting
on — the BLO-18858 duplicate-work defect, reintroduced through the review path that #1117
opened up.

New `assertPendingReviewRunOwnership` is the `in_review` counterpart of
assertCheckoutOwner, kept separate rather than widening that one: assertCheckoutOwner is
keyed on `in_progress` in eight places and owns the stale-lock adoption ladder.

The fence is deliberately narrow — live-locked AND `assigneeAgentId === actor` AND the
stage is pending. By the time control reaches it the actor may legitimately be a
mention-granted peer, a manager-chain actor, a recovery owner, a human, or a drifted
`currentParticipant`; none hold the checkout and all must still approve. Widening past
the assignee term would break the approval-by-comment path this issue exists to protect,
which is a worse regression than the bug.

AC3: a pending stage pinned to a `currentParticipant` whose id has drifted off
`assigneeAgentId` could never take the lock, so its already-authorized stage decision
raced whoever held it. The claim writes ONLY the lock columns — assignee, status and
startedAt are untouched — so it cannot become a back door to general issue ownership, and
the participant pin is re-asserted inside the UPDATE as a compare-and-swap.
@allyblockcast

allyblockcast Bot commented Aug 15, 2026

Copy link
Copy Markdown
Author

🔗 Paperclip issue: BLO-18858
🔗 Paperclip issue: BLO-22666

1 similar comment
@allyblockcast

allyblockcast Bot commented Aug 15, 2026

Copy link
Copy Markdown
Author

🔗 Paperclip issue: BLO-18858
🔗 Paperclip issue: BLO-22666

@allyblockcast

allyblockcast Bot commented Aug 15, 2026

Copy link
Copy Markdown
Author

Requesting a human review — @kkroo — because the automated reviewer is down, not slow

@kkroo asking you specifically because you authored #1117, which shipped the checkout half of BLO-22666. This PR is the remaining AC2 + AC3 half, re-cut on the substrate #1117 established. You have the most context on it of anyone.

Deliberately not requesting Ally. Ally is pinned to openai/gpt-5.6-terra, one of the two models in the exhausted codex BYOS pool (BLO-27123, critical, human-assigned). Measured this run:

codex successful requests 0/min, every hour since ~08:00Z (11h)
Ally's last review anywhere on this repo 2026-08-14T21:20:51Z (~23h ago)
sibling PRs with unanswered marker requests #1355, #1369, #1370, #1373, #1374

So Ally's silence here is not a per-PR delivery drop — I checked that first, and the repo-wide sweep is what ruled it out. Posting a <!-- paperclip:review-request --> marker would only enqueue another run that dies at the model call, so I have not posted one.

State at head 9ab20777a

  • 20/20 checks complete — 19 success, 1 skipped (Storybook visual regression, n/a for a server-only diff). verify, review, security-review all green.
  • mergeable: true, mergeable_state: clean, rebaseable: true, single commit, no merge commits.
  • Not in the merge queue; autoMergeRequest: null; reviewDecision empty (no required review on master).

Nothing mechanically blocks this merge — which is exactly why I am not merging it. The diff is a concurrency-critical authorization predicate that produced four consecutive Critical findings across 10 review rounds in its earlier incarnations (#821, #1120). Landing that unreviewed because the reviewer happens to be down is the wrong trade, and repo precedent for agent self-merge is not authorization on this particular surface.

What to look at, if you only have a few minutes

The whole risk is in how narrowly the fence arms. Every prior Critical came from widening it too far:

  • isForeignRunOfLockedPendingReview (routes/issues.ts) arms only when the row is live-locked and issue.assigneeAgentId === actorAgentId and the execution stage is pending — strictly the two-runs-of-one-agent case from BLO-18858.
  • It is a separate entry point from assertCheckoutOwner rather than a widening of it, because that one is keyed on in_progress in eight places and owns the stale-lock adoption ladder.
  • Mention-granted peers, manager-chain actors, recovery owners, and humans must still approve without holding the checkout. A regression there is worse than the bug. Four of the eleven new tests are negative controls for exactly those actors.
  • AC3's claim writes only the lock columns, so it cannot widen into general issue ownership; the participant pin is re-asserted inside the UPDATE as a CAS.

Each guard is verified load-bearing: neutering isForeignRunOfLockedPendingReview fails 3 tests, neutering allowExecutionStageParticipantClaim fails 1, and the four negative controls stay green under both.

No rush from my side — I have moved BLO-22666 to blocked behind BLO-27123 rather than leave it polling. Happy to take changes whenever you get to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant