Skip to content

Fix claim aging race and retained body hydration on 0.6 - #663

Merged
hardbyte merged 3 commits into
release/0.6.6from
brian/fix-aged-claim-hydration-0.6
Nov 7, 2026
Merged

hardbyte merged 3 commits into
release/0.6.6from
brian/fix-aged-claim-hydration-0.6

Conversation

@hardbyte

@hardbyte hardbyte commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

A claim can commit before its lane cursor advances. Physical aging could then tombstone its body, after which completion, cancellation or rescue rolled back and left the claim ring pinned.

Physical aging excludes claimed rows before its batch limit, locks claim heads in their existing order, and rechecks the selected identities in a fresh statement. Hydration matches the exact attempt and uses unique emitted physical-lane evidence to distinguish colliding bodies. Equally supported bodies fail explicitly.

Active point reads and callback reads now recover a retained tombstoned body when the normal body lookup misses. This fallback only reads; mutable attempt state and receipt closures remain transition work. Generic cancellation resolves the running attempt before a historical available replacement and shares the normal hydrated receipt transition.

When hydration selects a tombstoned source, recovery takes the affected claim heads with NOWAIT, rechecks all ready copies of that attempt and tombstones its old replacements. A fresh retry has a different ready run lease and remains eligible. Competing emitted copies or head contention abort the whole transaction. Conflicting historical terminal bodies use the existing self-contained representation so their physical priority and payload survive later retry and retention carry.

No schema changes or strong relation locks are added. Ordinary live-body reads and hydration keep their existing relation lookups. Historical recovery adds scoped attempt-ledger reads, weak relation locks and tombstone writes. Fresh mutation rechecks rely on READ COMMITTED isolation.

The soak harness fails on missing or ambiguous ready-body errors from parent Rust workers and claiming Rust children, and waits for child log readers before reconciliation. Default aging remains enabled. Operator-facing configuration and metric names are restored in the changelog. The main port is #664.

Validation:

  • Old-code failures reproduced deterministically, including a production-equivalent replacement segment and one-row claims after rescue.
  • Root and Python Rust workspaces pass formatting, Clippy with warnings denied and builds.
  • Sixty-one focused cases pass on each PG17/PG18: 41 affected-fixture and cancellation controls, plus 20 unchanged body/carry controls on identical product code. No ignored cases.
  • All 26 isolated native PG18 relation-lock controls match the prior candidate. The original 21 read/claim/complete controls do not exceed 0.6.13. Existing additional weak locks on retry and terminal admin mutations remain; these controls do not measure workload rates.
  • Current-head full CI is in progress; one pre-existing promotion-contention test failed on PG17 and is being investigated. The workload lock-pressure release gate remains outstanding.

Known limits: already-pruned bodies cannot be rebuilt; claims without unique emitted evidence retain the documented compatibility fallback. Worker completion using its already carried job body does not perform historical replacement retirement, so it can leave an old replacement claimable. Updated physical-aging callers prevent new replacements but cannot fence an older caller during rollout. Existing terminal-view and read/prune limitations remain.

@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8afc7e32-f014-43e1-9b27-77e25d194b60

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-07T02:55:41.412133Z 61db389 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61db389d19

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +10784 to +10787
let ambiguous: Vec<&ReadySnapshotRow> = ready_map
.values()
.filter(|rows| rows.len() > 1 && !rows.iter().any(|row| row.live_body))
.flatten()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Let emitted lane evidence override colliding live copies

When pre-fix physical aging raced a receipt claim and the destination priority lane allocated the same lane_seq as the source—commonly sequence 1 in two previously empty lanes—the tombstoned claimed source and its live replacement share this hydration key. This filter excludes any group containing a live row from the attempt-evidence lookup, so claimed_lanes remains empty and the later ranking selects the replacement despite the ledger identifying the tombstoned source. Rescue or completion then persists the replacement priority and can retry the job on the wrong physical lane; include these collision groups in the lookup and let emitted lane evidence select the claimed body.

Useful? React with 👍 / 👎.

@hardbyte
hardbyte merged commit 2681497 into release/0.6.6 Nov 7, 2026
77 of 78 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant