[codex] Make recovery updates atomic (BLO-18829)

[kkroo]

Thinking Path

• Paperclip is the open source control plane people use to manage AI agents for work.
• Its stranded-issue recovery path moves stalled work to a recovery state and wakes an owner.
• BLO-18829 found that recovery side effects could persist before the guarded issue-status compare-and-set had won.
• A concurrent status update could therefore leave recovery actions, quota monitors, or a wake marker for a transition that never committed.
• This master-based successor makes the source transition and its durable recovery state transactional, then performs external side effects only after commit.
• It supersedes the App-authored original with an independently authored branch suitable for the required review path.
• The result is no partial recovery state from a lost CAS, while retained wake work remains retryable.

Linked Issues or Issue Description

Supersedes #820.

Refs #820 and BLO-18829.

This folds the narrow durable-wake marker lifecycle correction from #1101. That PR is based on #820 rather than master, so it cannot serve as the required independently authored master-based successor.

What Changed

• Enlists recovery actions, provider quota monitors, issue-status CAS, and durable recovery-wake outbox writes in the same transaction.
• Uses the expected status plus exact updated_at token for the recovery CAS and rolls back all related writes when it loses.
• Defers comments, activity, and wake dispatch until after a successful commit.
• Preserves transaction-safe recovery-action upserts and correctly distinguishes non-evidence park faults from an explicit blocked transition.
• Deletes a recovered durable-wake marker after successful dispatch, retains failed markers for retry, and covers terminal retirement behavior.
• Adds regression coverage for recovery CAS rollback, post-commit wake handling, and issue-service status recovery paths.

Verification

Passed locally against current master:

• pnpm --filter @paperclipai/server typecheck
• pnpm --filter @paperclipai/server exec vitest run --no-file-parallelism src/__tests__/issue-recovery-actions.test.ts
• pnpm --filter @paperclipai/server exec vitest run --no-file-parallelism src/__tests__/issues-service.test.ts
• pnpm --filter @paperclipai/server exec vitest run --no-file-parallelism src/__tests__/heartbeat-process-recovery.test.ts
• git diff --check refs/remotes/https-origin/master...HEAD

The heartbeat suite emitted the existing test-environment warning for an unavailable optional external adapter; it did not fail the suite.

Risks

Medium: this changes transaction boundaries in stranded-issue recovery. The main failure mode would be an incorrectly retained or removed wake marker; focused tests cover lost CAS rollback, retryable failed dispatch, and terminal cleanup. No migration or lockfile change is included.

Model Used

OpenAI Codex (GPT-5), using repository inspection, local code changes, GitHub API/CLI access, and focused test execution.

Checklist

• I have included a thinking path that traces from project context to this change
• I have specified the model used (with version and capability details)
• I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
• I have searched GitHub for duplicate or related PRs and linked them above
• I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template
• I have run tests locally and they pass
• I have added or updated tests where applicable
• If this change affects the UI, I have included before/after screenshots — not applicable; server-only
• I have updated relevant documentation to reflect my changes — not applicable; no public contract changed
• I have considered and documented any risks above
• All Paperclip CI gates are green
• Greptile is 5/5 with no open P2s, recommendations, or follow-ups
• I will address all Greptile and reviewer comments before requesting merge

[allyblockcast]

🔗 Paperclip issue: BLO-18829

[allyblockcast]

🔗 Paperclip issue: BLO-18829

[kkroo]

Closing as stale — this draft can no longer be landed as a diff.

Measured against master ac91e966:

• 808 commits behind
• 5 conflicting file(s) (git merge-tree --write-tree)

At that distance the surrounding code has moved out from under the patch, so "resolve the conflicts" would mean rewriting it against a codebase it was never written for — not merging it. Reopening is cheap if the branch is still wanted; the commits are untouched.

Closing the PR does not close the underlying issue. Where this PR referenced a BLO issue, that issue remains the record of intent and the place to re-implement from, against current master.

No judgement here on whether the original problem still exists — I did not verify that per-PR, and it should be re-checked before any re-implementation, since some of these may already have been fixed independently.