Skip to content

Native: run_in_transaction refuses to replay a block that already committed part of its work (P-03) - #527

Merged
tae898 merged 2 commits into
native-previewfrom
fix/p03-native-run-in-tx-partial-commit
Oct 25, 2026
Merged

tae898 merged 2 commits into
native-previewfrom
fix/p03-native-run-in-tx-partial-commit

Conversation

@tae898

@tae898 tae898 commented Oct 25, 2026 •

Copy link
Copy Markdown

Backlog row P-03. A correctness fix for the native run_in_transaction, based on native-preview and independent of #522. It goes first: #522 (4 -> 3 crossings) rewrites the same lines, so #522 is rebased onto this branch and carries the fix; this PR can merge on its own and does not wait for the performance change.

Bug

The native run_in_transaction has a guard that refuses a retry when the block already committed part of its work (a statement with a BATCH n boundary, or a commit() inside the block): the rollback cannot take that part back, so a replay applies it twice. The guard was written as try: if <commit count> != commits_at_start: raise followed by except ArcadeDBError: pass. The raise re-raises the retryable ArcadeDBError of the failed attempt, and the except ArcadeDBError: pass of the same try catches it. The guard never fired, and the block was replayed. The JVM core (core.py) has no such handler and refuses.

Reproduced

A real conflict (a second thread commits to a record the block also wrote), 25 rows at v = 0, the block runs UPDATE R SET v = v + 1 with a publishing step, then conflicts on its first attempt. Native library nnext, before this change:

publishing step attempts rows with v = 2 sum(v) (single application: 20 or 25)
UPDATE ... BATCH 10 2 (replayed, returned success) 20 45
commit() + begin() inside the block 2 (replayed, returned success) 25 50

Fix

Only the read of the commit-count witness may fail quietly (no witness: retry, as the JVM does). The comparison and the raise now sit outside that try, so the original conflict escapes after the rollback, as on the JVM wheel. The contract is added to the run_in_transaction API doc.

Tests

tests/test_run_in_transaction_partial_commit.py has no Java class and no fabricated error, so one file runs on both wheels: the two publishing steps above (single application, one attempt, the conflict raised, no transaction left open) and a block that committed nothing, which is still retried once and succeeds.

Local checks

  • native (nnext library), native-preview without this change: the two partial-commit tests fail (red), the retry-still-works test passes
  • native (nnext library), this branch: the new file, test_core.py, test_run_in_transaction_retry.py and test_interrupt_hygiene.py: 62 passed, 9 skipped
  • JVM wheel wheel-main-78e97b1cff (main): the new file, 3 passed (parity, no change to the JVM side)
  • black, isort and the other pre-commit hooks on the touched files: clean; bandit on _native/database.py: no findings
  • Not run: the full suite, CI, timed runs, a native rebuild (Python only)

🤖 Generated with Claude Code

…mitted part of its work (P-03)

The guard that refuses a retry after a BATCH boundary or a commit() inside the block sat in a try whose own `except ArcadeDBError: pass` swallowed the `raise` it was written to make, so the block was replayed and its durable part applied twice. Only the read of the commit-count witness may fail quietly; the raise now sits outside that try. The JVM core already refused.

The new test uses a real conflict (a second thread commits to a record the block also wrote), so it runs on both wheels: red on the native wheel before this change (20 of 25 rows incremented twice after a BATCH boundary, all 25 after a commit() inside the block), green after, green on the JVM wheel. The contract is added to the run_in_transaction API doc.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 force-pushed the fix/p03-native-run-in-tx-partial-commit branch from 216bf0a to 514b225 Compare October 25, 2026 08:32
@tae898
tae898 changed the base branch from perf/native-tx-crossings to native-preview October 25, 2026 08:32
@tae898
tae898 marked this pull request as ready for review October 25, 2026 09:12
…e text)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 3d03ae7 into native-preview Oct 25, 2026
42 of 44 checks passed
@tae898
tae898 deleted the fix/p03-native-run-in-tx-partial-commit branch October 25, 2026 12:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant