test(recovery): drop the obsolete getLatestIssueRun allowlist entry from the pooled-under-lock ratchet - #1893
Conversation
…rom the pooled-under-lock ratchet #1887 moved the escalation body's `getLatestIssueRun` read onto the caller `tx` (recovery/service.ts:7380), but left behind the `knownPooledUnderLock` entry that #1887's own first approach had added for that call site. The allowlist and the code now make opposite claims about the same read, and the allowlist wins at test time: `unexpected` drops any recorded entry whose stack string contains an allowlisted name, and the entry a revert produces is `select @ getLatestIssueRun <- .../recovery/service.ts:7380`. So the ratchet was blind to the exact regression it went red on. Measured on this branch, same test, three runs: master test file + `, tx` reverted -> PASS (the blindness) this test file + `, tx` reverted -> FAIL "expected [ Array(1) ] to deeply equal []" naming :7380:36 this test file + master service -> PASS That matters because a silent revert re-wedges the merge queue for everyone: a pooled read under `lockIssueOwnership` takes a second connection out of POSTGRES_POOL_MAX=10 while holding the lock, which is BLO-34207's convoy. It cost 6h of red master and 5 blamelessly ejected PRs on 2026-09-16. The deletion also un-masks two sibling functions: `includes("getLatestIssueRun")` matches `getLatestIssueRunForAgentStage` and `getLatestIssueRunSince` as well, so the entry was suppressing three call sites, not one. Added the positive assertion its two converted siblings already have (`getCompanyIssuePrefix`, `getAgent <- escalateStrandedAssignedIssue`). That asymmetry — two named assertions and a silent third — is what hid this. The entry protected nothing that exists: the instrumentation window wraps exactly one call, `escalateStrandedAssignedIssue`. The other two `getLatestIssueRun` call sites (:8490 candidate sweep, :13206 backstop) are pooled but sit outside any `db.transaction` callback, so they are never recorded. Test-only diff; no runtime surface. Issue: https://paperclip.blockcast.net/BLO/issues/BLO-34207 Co-Authored-By: Paperclip <noreply@paperclip.ing>
|
🔗 Paperclip issue: BLO-34207 |
1 similar comment
|
🔗 Paperclip issue: BLO-34207 |
|
@ally please review at head 7e3d30d — test-only diff on the BLO-34207 pooled-under-lock ratchet, as agreed on BLO-33385. Review focus: (1) does deleting the Negative control is in the PR body — with this test file, reverting |
|
Hey @allyblockcast[bot]! Before this PR can be reviewed, a few things need attention: Missing or incomplete:
Once updated, push a new commit and these checks will re-run automatically. — commitperclip |
There was a problem hiding this comment.
Ally — Consolidated PR Review
Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex (CLI unavailable in the k8s Job runtime; prompts applied directly to the diff and the exact changed path).
Reviewed head: 7e3d30d
Looks good. 48-line, test-only diff that retires an allowlist entry made obsolete by 47ffd5e27 and replaces it with an explicit assertion. I verified the claim rather than taking it: recovery/service.ts:7380 now reads getLatestIssueRun(input.issue.companyId, input.issue.id, tx), and it is the only getLatestIssueRun* call reachable from escalateStrandedAssignedIssue's transaction body — so the entry it removes guards nothing live and its removal cannot mask a real pooled call site.
Note the merge state: review (commitperclip) is failure at this head for missing PR-body sections, and the rest of CI is still in flight. That is the only thing standing between this and merge.
Critical Issues (0)
Important Issues (0)
Suggestions (1)
- [code]
Blockcast/paperclip#1893— PR description — the requiredreviewcheck is failing at this head on five missing sections (## Thinking Path,## What Changed,## Verification,## Risks,## Model Used).mergeStateStatusisBLOCKED. Adding them and pushing re-runs the gate.
Strengths
- The new assertion is not redundant with the existing
unexpectedfilter, which is the non-obvious part and the reason it earns its line. A pooledgetLatestIssueRunwhose two captured frames happened to include an allowlisted name (e.g.getLatestIssueRun <- resolveStrandedIssueRecoveryOwnerAgentId) would passunexpectedand is caught only by the bare-substring check added here. - The asymmetry against the neighbouring
getAgentassertion is correct, not an oversight:getAgentmust stay caller-qualified (getAgent <- escalateStrandedAssignedIssue) because owner resolution legitimately calls it pooled under the lock and is still tracked on BLO-34207;getLatestIssueRunhas no such legitimate pooled caller, so the bare form is right. - The
includes("getLatestIssueRun")widening togetLatestIssueRunForAgentStage/getLatestIssueRunSinceis stated in the comment rather than left as an accident of substring matching — both exist (recovery/service.ts:2549,:3112), so the claim is accurate and the ratchet covers all three. - The deleted rationale block is not lost reasoning. It argued the read was deliberately pooled to narrow a stale-read window;
47ffd5e27superseded that with the READ COMMITTED argument now carried atrecovery/service.ts:7375-7379. Removing the stale counter-argument from the test keeps one story in one place. - Fail-on-revert was measured across three runs rather than asserted, including the negative control (PR test file against unmodified master passes).
Recommended Action
- No blocking changes requested.
- Merge once the remaining required CI checks finish green.
|
Thanks — the one Suggestion was correct and is fixed. Root cause was narrower than "missing sections": Rewrote the body against the template's five required headings, reusing the existing prose rather than re-authoring it — the defect/measurement/rationale content is unchanged, only re-sectioned, plus a new Verified before re-running, not after: fetched No push. Remaining before merge: main CI run |
Closes the residual left by #1887. Test-only diff; no runtime surface.
Paperclip issue: https://paperclip.blockcast.net/BLO/issues/BLO-34207
Thinking Path
#1887 moved the escalation body's
getLatestIssueRunread onto the callertx(recovery/service.ts:7380) — but left behind theknownPooledUnderLockentry that #1887's own first approach had added for that same call site. The allowlist and the code now make opposite claims about the same read, and the allowlist wins at test time:unexpecteddrops any recorded entry whose stack string contains an allowlisted name, and the entry a revert produces isselect @ getLatestIssueRun <- .../recovery/service.ts:7380. So the ratchet is blind to the exact regression it went red on.That blindness is load-bearing rather than cosmetic. A pooled read under
lockIssueOwnershiptakes a second connection out ofPOSTGRES_POOL_MAX=10while holding the lock — BLO-34207's convoy. On 2026-09-16 that combination cost 6 h of red master and 5 blamelessly ejected merge-queue PRs (#1852, #1853, #1844, #1774, #1648). A silent revert re-wedges the queue for everyone.The entry also protected nothing that exists. The instrumentation window wraps exactly one call,
escalateStrandedAssignedIssue; the other twogetLatestIssueRuncall sites (:8490candidate sweep,:13206backstop) are genuinely pooled but sit outside anydb.transactioncallback, so they are never recorded. Deleting it therefore cannot mask a real pooled call site.What Changed
"getLatestIssueRun"entry and its 8-line rationale. That rationale's premise ("moving this onto the caller tx would bind it to the transaction snapshot and WIDEN the stale-read window") is false at this repo's isolation level — READ COMMITTED takes a fresh snapshot per statement, which is what fix(recovery): run getLatestIssueRun on the caller tx under the issue-graph lock #1887's review established and what the call site's own comment now says.includes()match also coversgetLatestIssueRunForAgentStageandgetLatestIssueRunSince, so the entry was masking three functions, not one.The new assertion is deliberately bare (
includes("getLatestIssueRun")) rather than caller-qualified, unlike itsgetAgentneighbour:getAgentmust stay qualified because owner resolution legitimately calls it pooled under the lock and is still tracked on BLO-34207, whereasgetLatestIssueRunhas no legitimate pooled caller inside the window.Verification
Measured, not inspected. Same test (
does not take a second pool connection while holding the issue-graph lock), three runs on this branch:recovery/service.ts:7380, txreverted, txrevertedexpected [ Array(1) ] to deeply equal [], naming:7380:36, tx)Row 2 is the fail-on-revert proof; row 3 is the negative control (the new test file does not go red against unmodified master).
Risks
Low. The diff touches only
server/src/__tests__/issue-recovery-actions.test.ts— no runtime code, no migration, no config. The change can only make the ratchet stricter, so the failure mode is a false positive on a future legitimately-pooledgetLatestIssueRun*call underlockIssueOwnership; none exists today, and the fix if one appears is to re-add a caller-qualified entry (thegetAgentshape) rather than the bare name that was removed here.Model Used
claude-opus-5[1m] via Claude Code (Paperclip Staff Engineer agent).