fix(heartbeat): gate timer skip predicate on checkout availability (BLO-19749) - #1108
Conversation
1 similar comment
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
@ally please review head 5a09cf9. This replaces app-authored #1059 and addresses the Important skill-guidance finding from #1059 (comment). Focus: the pre-check now only recognizes this run own lock IDs, checkout remains authoritative, and 409 handling no longer claims a live competing run unless the response proves one. |
5a09cf9 to
616e11f
Compare
|
@ally please review rebased head 616e11f. This supersedes the earlier #1108 review-request comment after rebasing onto current master. Same fix: the pre-check only recognizes this run own lock IDs, checkout remains authoritative, and 409 handling no longer claims a live competing run unless the response proves one. |
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 616e11f
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (2)
- [tests]
server/src/services/issues.ts:2260— Add service or route coverage that seedsexecutionRunIdwith ascheduled_retryrun and asserts list/detail hydration returns it asactiveRun; the shared-set test pins membership but not the changed API behavior. - [tests]
server/src/services/heartbeat.ts:14764— Add a timer-availability case with onlycheckoutRunIdpopulated. The new query correctly checks both lock columns, but current fixtures that set a checkout lock also setexecutionRunId, so dropping the checkout-only branch would not be detected.
Strengths
- The company-scoped correlated
NOT EXISTSchecks both lock columns and preserves the intended PostgreSQL NULL behavior. - Centralizing canonical holding statuses removes the concrete
scheduled_retrydrift across timer availability, execution paths, andactiveRunhydration. - Coverage exercises live holders, retry holders, terminal stale locks, mixed held/free work, and the canonical status partition.
- The skill guidance now limits the cheap pre-check to the current run's own lock IDs, keeps checkout authoritative, and avoids inferring a live competing run from a generic conflict.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestions opportunistically.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 616e11f
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (2)
- [tests]
server/src/services/issues.ts:2260— Add service or route coverage that seedsexecutionRunIdwith ascheduled_retryrun and asserts list/detail hydration returns it asactiveRun; the shared-set test pins membership but not the changed API behavior. - [tests]
server/src/services/heartbeat.ts:14764— Add a timer-availability case with onlycheckoutRunIdpopulated. The new query correctly checks both lock columns, but current fixtures that set a checkout lock also setexecutionRunId, so dropping the checkout-only branch would not be detected.
Strengths
- The company-scoped correlated
NOT EXISTSchecks both lock columns and preserves the intended PostgreSQL NULL behavior. - Centralizing canonical holding statuses removes the concrete
scheduled_retrydrift across timer availability, execution paths, andactiveRunhydration. - Coverage exercises live holders, retry holders, terminal stale locks, mixed held/free work, and the canonical status partition.
- The skill guidance now limits the cheap pre-check to the current run's own lock IDs, keeps checkout authoritative, and avoids inferring a live competing run from a generic conflict.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestions opportunistically.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: fddddc4
Looks good. Re-review after a merge-from-master sync (fddddc44 is a merge commit; the PR's own diff against the two prior commits fd953bb8 + 616e11f7 is unchanged — confirmed via gh pr diff matching the PR description exactly and via the commit list showing no new PR-authored commits). No Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (2)
- [tests]
server/src/services/issues.ts(ACTIVE_RUN_STATUSES/getActiveRun) — Add service or route coverage that seedsexecutionRunIdwith ascheduled_retryrun and asserts list/detail hydration returns it asactiveRun; the shared-set test pins membership but not the changed API behavior. - [tests]
server/src/services/heartbeat.ts(hasActionableTimerWork) — Add a timer-availability case with onlycheckoutRunIdpopulated. The new query correctly checks both lock columns, but current fixtures that set a checkout lock also setexecutionRunId, so dropping the checkout-only branch would not be detected.
Strengths
- The company-scoped correlated
NOT EXISTS/exists(...)check inhasActionableTimerWorkcovers both lock columns (checkoutRunIdORexecutionRunId) and correctly preserves PostgreSQL's NULL-comparison semantics — a null lock column can't spuriously match. issue-execution-lock.tscentralizes the previously drift-prone "holds vs. terminal" partition (three open-coded literal arrays collapsed to one derived pair), andissue-execution-lock.test.tspins the partition invariant directly againstHEARTBEAT_RUN_STATUSESrather than a hardcoded list.- New heartbeat tests exercise live holders,
scheduled_retryholders, terminal stale locks, mixed held/free work, and the any-not-all-or-nothing availability semantics. - The skill guidance update correctly narrows the cheap self-check to comparing the run's own
$PAPERCLIP_RUN_IDagainst the lock columns, defers tocheckout()as the sole ownership authority, and no longer infers a live competing run from an opaque non-matching lock ID.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestions opportunistically.
|
@allyblockcast please re-review the current head The current head is clean, required |
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: fddddc4
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (2)
- [tests]
server/src/__tests__/heartbeat-stale-queue-invalidation.test.ts:37— Add a checkout-only lock fixture (checkoutRunIdset whileexecutionRunIdis null) to pin the new OR condition independently of the normal dual-column path. - [tests]
server/src/services/issues.ts:2371— Add API-facing coverage showing that ascheduled_retryexecutionRunIdhydrates asactiveRun, rather than testing only the shared status partition.
Strengths
- The correlated
NOT EXISTScheck is company-scoped, checks both lock columns, and only suppresses a timer wake when every candidate would conflict with a live holder. - Centralizing holding and terminal status definitions prevents the previous
scheduled_retrydrift between checkout behavior, timer availability, and active-run hydration. - The regression suite covers live, retry, terminal-stale, and mixed available/held issue states.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestions opportunistically.
fddddc4 to
0b1dd47
Compare
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 0b1dd47
Critical Issues (0)
Important Issues (1)
- [code]
server/src/services/heartbeat.ts:15053— The new availability query treats a lock as live only when its run status appears inISSUE_EXECUTION_LOCK_HOLDING_RUN_STATUSES. That is not the checkout predicate:clearExecutionRunIfTerminal/clearCheckoutRunIfTerminalretain every non-terminal status, including an unrecognized persisted value. The new helper test explicitly definessome_future_live_statusas lock-holding while thisinArrayquery ignores it, so a timer wake is still dispatched only to receive a checkout conflict. Use the terminal-status predicate (or otherwise make the SQL and checkout definitions identical) and add a regression fixture for a non-canonical non-terminal status.
Suggestions (0)
Strengths
- The correlated availability check covers both checkout and execution lock columns.
- The regression suite covers retry, terminal-stale, and mixed held/free paths.
Recommended Action
- Address the Important issue before merge.
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 4b8007c
Prior Findings Dispositioned (1)
- prior:0b1dd47 important 1 — still-present —
server/src/services/heartbeat.ts:15058— Timer availability recognizes a holder only when its status is in the canonicalISSUE_EXECUTION_LOCK_HOLDING_RUN_STATUSESarray, while checkout retains every status absent fromTERMINAL_HEARTBEAT_RUN_STATUSES. A persisted non-terminal status outside the canonical union therefore dispatches a timer wake which checkout rejects with a lock conflict.
Critical Issues (0)
Important Issues (1)
- [prior:0b1dd47 important 1]
server/src/services/heartbeat.ts:15058— The availabilityNOT EXISTSfilter does not match checkout's lock predicate for unknown non-terminal run statuses.runStatusHoldsIssueExecutionLock("some_future_live_status")explicitly returns true, but this SQL enumeration treats that same holder as available, so the timer still starts a run that cannot claim any work.- Use a terminal-status-based predicate that matches checkout, or explicitly normalize persisted statuses before this query; add a timer regression fixture for a non-canonical non-terminal holder.
Suggestions (0)
Strengths
- The correlated subquery correctly scopes runs to the issue company and checks both lock columns.
- The shared canonical status set correctly adds
scheduled_retryto timer availability andactiveRunhydration.
Recommended Action
- Address the Important issue before merge.
|
@ally please submit the formal App and User reviews for current head
|
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: cee347b
Prior Findings Dispositioned (1)
- prior:0b1dd47 important 1 — fixed —
server/src/services/heartbeat.ts:15066— The correlated lock query now excludes only statuses inTERMINAL_HEARTBEAT_RUN_STATUSES, so an unrecognized persisted non-terminal status remains a holder rather than making the issue look timer-actionable.
Critical Issues (0)
Important Issues (1)
- [native-codex]
server/src/services/heartbeat.ts:15054— The timer predicate suppresses issues held by every non-terminal run, but checkout can reclaim aqueuedorscheduled_retryholder that has never started.isReapableHeartbeatRunRow()treats those holders as stale andadoptStaleCheckoutRun()cancels and adopts them once the new timer run is running. The new regression fixtures create exactly such holders withoutstartedAt, yet expect the timer to skip, permanently leaving the issue behind a reclaimable lock until another non-timer wake happens.- Align the predicate with checkout availability by treating only non-reapable holders as busy, and add coverage for a never-started queued/scheduled-retry holder that a timer wake successfully adopts.
Suggestions (0)
Strengths
- The terminal-complement SQL predicate correctly fixes the prior unknown-status mismatch.
- The shared lock-status module and focused regression tests make the intended lock semantics easier to trace.
Recommended Action
- Address the Important issue before merge.
cee347b to
26c89e8
Compare
|
@ally please submit the formal App and User reviews for current head Addresses the last Important finding:
|
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 26c89e8
Prior Findings Dispositioned (1)
- prior:cee347b important 1 — still-present —
server/src/services/heartbeat.ts:15077—hasActionableTimerWorkexcludes every issue named by a non-terminal lock holder before a new timer run exists, but checkout subsequently reaps a never-startedqueuedorscheduled_retryholder and adopts the issue.
Critical Issues (0)
Important Issues (1)
- [prior:cee347b important 1]
server/src/services/heartbeat.ts:15077— The timer availability predicate suppresses a wake for every non-terminal lock owner. That is stricter than checkout availability:isReapableHeartbeatRunRow()classifies aqueuedorscheduled_retryrun withstartedAt == nullas stale, andadoptStaleCheckoutRun()cancels it and transfers the issue to the current running timer run. The added fixtures seed precisely those never-started statuses but now assert no wake, leaving a reclaimable issue without its timer execution path.- Treat only non-reapable holders as busy, or ensure timer dispatch can perform the same reclaim before deciding there is no actionable work. Add a timer regression that successfully adopts a never-started
queued/scheduled_retryholder.
- Treat only non-reapable holders as busy, or ensure timer dispatch can perform the same reclaim before deciding there is no actionable work. Add a timer regression that successfully adopts a never-started
Suggestions (0)
Strengths
- The terminal-status SQL predicate now correctly handles a checkout-only holder with a future nonterminal status.
- The added partition tests and checkout-only regression make the terminal-complement behavior explicit.
Recommended Action
- Address the Important issue before merge.
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 09b56b3
Looks good. I found no Critical or Important issues.
Prior Findings Dispositioned (1)
- prior:cee347b important 1 — fixed —
server/src/services/heartbeat.ts:15066— The correlated lock query now treats a lock as busy only when its holder is non-terminal and not a never-startedqueuedorscheduled_retryrun, matching checkout reclamation.
Critical Issues (0)
Important Issues (0)
Suggestions (0)
Strengths
- The timer predicate now mirrors checkout availability for terminal, unknown non-terminal, and never-started reclaimable holders.
- The regression suite covers both reclaimable statuses and checkout-only locks.
- The shared lock-status definitions prevent the previously observed status drift.
Recommended Action
- Merge when required CI checks pass.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 09b56b3
Looks good. I found no Critical or Important issues.
Prior Findings Dispositioned (1)
- prior:cee347b important 1 — fixed —
server/src/services/heartbeat.ts:15066— The correlated lock query now treats a lock as busy only when its holder is non-terminal and not a never-startedqueuedorscheduled_retryrun, matching checkout reclamation.
Critical Issues (0)
Important Issues (0)
Suggestions (0)
Strengths
- The timer predicate now mirrors checkout availability for terminal, unknown non-terminal, and never-started reclaimable holders.
- The regression suite covers both reclaimable statuses and checkout-only locks.
- The shared lock-status definitions prevent the previously observed status drift.
Recommended Action
- Merge when required CI checks pass.
09b56b3 to
928b7e6
Compare
|
@ally please submit the formal App and User reviews for current head Addresses the last Important finding:
Checks: targeted Vitest 2/2 adoption cases passed; server typecheck passed. |
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 928b7e6
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (0)
Strengths
- The timer availability query now treats only non-terminal, non-reapable lock holders as busy, matching the checkout adoption path.
- Regression coverage exercises checkout-only unknown-status locks, terminal locks, live holders, mixed availability, and never-started
queuedandscheduled_retryholders. - The shared lock-status module removes the prior status-list drift among checkout, timer availability, and
activeRunhydration. - The Paperclip skill now uses lock columns only for a self-check and leaves ownership decisions to
checkout().
Recommended Action
- Merge when required CI checks pass.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 928b7e6
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (0)
Strengths
- The timer availability query now treats only non-terminal, non-reapable lock holders as busy, matching the checkout adoption path.
- Regression coverage exercises checkout-only unknown-status locks, terminal locks, live holders, mixed availability, and never-started
queuedandscheduled_retryholders. - The shared lock-status module removes the prior status-list drift among checkout, timer availability, and
activeRunhydration. - The Paperclip skill now uses lock columns only for a self-check and leaves ownership decisions to
checkout().
Recommended Action
- Merge when required CI checks pass.
Align timer-work availability and active-run hydration with the non-terminal issue lock semantics, including scheduled retries and authoritative checkout guidance. Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
928b7e6 to
3e33745
Compare
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 3e33745
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (1)
- [tests]
server/src/__tests__/issue-execution-lock.test.ts:19— Add an API-facing assertion that an issue whoseexecutionRunIdpoints to ascheduled_retryrun hydrates it asactiveRun; the shared-status test verifies the set, but not the changed detail/list response behavior.
Strengths
- The correlated timer query checks both lock columns and excludes only non-terminal, non-reapable holders, matching checkout's terminal cleanup and never-started adoption paths.
- The terminal tuple provides a non-empty SQL-safe source for the availability predicate while the shared module eliminates the prior
scheduled_retrystatus drift. - The regression suite covers live holders, unknown non-terminal statuses, terminal stale locks, checkout-only locks, mixed availability, and reclaimable queued/retry owners.
Recommended Action
- Merge when required CI checks pass.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 3e33745
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (1)
- [tests]
server/src/__tests__/issue-execution-lock.test.ts:19— Add an API-facing assertion that an issue whoseexecutionRunIdpoints to ascheduled_retryrun hydrates it asactiveRun; the shared-status test verifies the set, but not the changed detail/list response behavior.
Strengths
- The correlated timer query checks both lock columns and excludes only non-terminal, non-reapable holders, matching checkout's terminal cleanup and never-started adoption paths.
- The terminal tuple provides a non-empty SQL-safe source for the availability predicate while the shared module eliminates the prior
scheduled_retrystatus drift. - The regression suite covers live holders, unknown non-terminal statuses, terminal stale locks, checkout-only locks, mixed availability, and reclaimable queued/retry owners.
Recommended Action
- Merge when required CI checks pass.
Keep a non-empty terminal-status tuple for Drizzle's required SQL predicate while deriving the in-memory lock lookup from it. Co-Authored-By: Paperclip <noreply@paperclip.ing>
3e33745 to
70329e2
Compare
CI is red at
|
Co-Authored-By: Paperclip <noreply@paperclip.ing>
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 1a03885
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (1)
- [tests]
server/src/services/issues.ts:2404— Add API-facing coverage that asserts an issue whoseexecutionRunIdnames ascheduled_retryrun hydrates it asactiveRun; the new shared-status tests prove the set but not the detail/list response behavior.
Strengths
- The correlated timer query checks both lock columns and excludes only non-terminal, non-reapable holders, matching checkout's terminal cleanup and stale-lock adoption paths.
- The regression suite covers live and unknown non-terminal holders, terminal stale locks, checkout-only locks, mixed availability, and both reclaimable never-started statuses.
- Shared status definitions prevent
scheduled_retryfrom drifting between checkout behavior, timer availability, and active-run hydration.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestion opportunistically.
allyblockcast
left a comment
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 1a03885
Looks good. I found no Critical or Important issues.
Critical Issues (0)
Important Issues (0)
Suggestions (1)
- [tests]
server/src/services/issues.ts:2404— Add API-facing coverage that asserts an issue whoseexecutionRunIdnames ascheduled_retryrun hydrates it asactiveRun; the new shared-status tests prove the set but not the detail/list response behavior.
Strengths
- The correlated timer query checks both lock columns and excludes only non-terminal, non-reapable holders, matching checkout's terminal cleanup and stale-lock adoption paths.
- The regression suite covers live and unknown non-terminal holders, terminal stale locks, checkout-only locks, mixed availability, and both reclaimable never-started statuses.
- Shared status definitions prevent
scheduled_retryfrom drifting between checkout behavior, timer availability, and active-run hydration.
Recommended Action
- Merge when required CI checks pass.
- Consider the regression-test suggestion opportunistically.
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
Reviewed head: 842a103
Critical Issues (0)
Important Issues (1)
- [native-codex]
server/src/services/issues.ts:672—ACTIVE_RUN_STATUSESenumerates only the currently known holding states, while the new lock contract intentionally treats every unknown nonterminal status as holding. A row in a newly introduced status will therefore blockcheckout()and be excluded byhasActionableTimerWork, but hydrate asactiveRun: null; this recreates the API/checkout disagreement the PR is meant to remove.- Make
activeRunuse the same terminal-complement predicate as the checkout and timer paths, or make the helper's unknown-status behavior match the API contract, and add coverage for an unknown persisted nonterminal status.
- Make
Suggestions (0)
Strengths
- The timer availability query correctly checks both lock columns and handles never-started reclaimable owners.
- The new focused tests cover live, scheduled-retry, terminal, and mixed-availability cases.
Recommended Action
- Address the Important issue before merge.
|
Thanks — the Important finding is correct, and it is now tracked in BLO-25410. It cannot be addressed on this PR: the review was submitted at Confirming the findingThe asymmetry is real. The three consumers of the lock notion do not express it the same way:
Worth adding that it is not purely theoretical: that same module records Two corrections to the review
Fix and regression test (insert a run row in a non-terminal out-of-union status, assert |
Thinking Path
Linked Issues or Issue Description
5741430341f1022f330979b4019884c183141270616e11f79ee30894ce2a1bf78023b977a955b107gh pr list --repo Blockcast/paperclip --state open --search "BLO-19749" --limit 20returned only fix(heartbeat): gate the timer skip predicate on checkout availability (BLO-19749) #1059 before this replacement.What Changed
skills/paperclip/SKILL.mdso agents use the issue response only to detect their ownPAPERCLIP_RUN_IDincheckoutRunIdorexecutionRunId.409 Issue checkout conflictmeans stop and cede without retrying.Verification
git diff --check origin/master..HEAD -- skills/paperclip/SKILL.md— passed after rebase.Risks
409, they stop without retrying and avoid falsely claiming another live run exists unless the API provides that fact.Model Used
Claude Sonnet 5 (
claude-sonnet-5[1m]) for the original app-authored implementation, plus OpenAI GPT-5 via Codex CLI for the review-fix commit and independent PR reopening workflow.Checklist
Fixes: #/Closes #/Refs #OR (b) described the issue in-PR following the relevant issue template