Repository navigation
Conversation
…mitted part of its work (P-03) The guard that refuses a retry after a BATCH boundary or a commit() inside the block sat in a try whose own `except ArcadeDBError: pass` swallowed the `raise` it was written to make, so the block was replayed and its durable part applied twice. Only the read of the commit-count witness may fail quietly; the raise now sits outside that try. The JVM core already refused. The new test uses a real conflict (a second thread commits to a record the block also wrote), so it runs on both wheels: red on the native wheel before this change (20 of 25 rows incremented twice after a BATCH boundary, all 25 after a commit() inside the block), green after, green on the JVM wheel. The contract is added to the run_in_transaction API doc. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Oct 25, 2026
tae898
force-pushed
the
perf/native-tx-crossings
branch
from
October 25, 2026 08:35
ae3cb02 to
0e56306
Compare
…e text) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gin reply, and the unwind reads no state it already has
run_in_transaction made 4 library crossings on the happy path (mark, begin, commit mark, commit) although the begin reply already carries {"active","depth","commits"}; the commit-count witness now comes from that reply: 3 crossings (the mark must stay before begin, see the docstring).
The unwind after a failed block (_end_transactions_opened_since) asked is_transaction_active() before every rollback and once more at the end, and the retry check asked for the commit count again: every rollback reply carries the same state, so the unwind returns the state of its last crossing and the retry check reads the commit count from it (a failed crossing falls back to a read). A failed block: 6 crossings -> 4; a refused attempt that is retried: 11 -> 7.
Tests first: test_native_tx_crossings.py counts lib.call invocations (sequence of arcadedb_n_tx operations) for run_in_transaction, `with db.transaction()` and the exception and retry paths, and pins the behaviour the marks exist for (nested block leaves the caller's transaction open, refused attempt rolled back before the retry, duplicated key retried once). Counts only; no speed claim.
The unwind and begin now go through two named seams, _begin_state() and _rollback_state() (begin() and rollback() are thin wrappers over them). test_interrupt_hygiene.py patched Database.begin and Database.rollback, which run_in_transaction no longer calls on the native Database: two tests failed (a refused begin, a failing rollback) and three interrupt tests failed with "DID NOT RAISE KeyboardInterrupt", which conftest.py turns into a silent "NA-JVM" skip. The tests now patch the seam the active backend's run_in_transaction really calls (begin/rollback on the JPype Database, _begin_state/_rollback_state on the native one) and check what they checked before; the one test with no native equivalent (an interrupt between begin and a separate commit-mark read, which the begin reply made one crossing) skips with a stated reason.
Based on the P-03 fix (the commit-count witness check raises outside its try), so the retry check here reads the witness from the unwind state and keeps that fix.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
force-pushed
the
perf/native-tx-crossings
branch
from
October 25, 2026 09:31
0e56306 to
aeacaa0
Compare
8 of 10 tasks
Author
|
Closing here: the bindings moved to ArcadeData/arcadedb-python. This change has no counterpart there yet and no queued A/B. If it is wanted, it goes there as its own PR: it is now a line in the tracking issue ArcadeData/arcadedb-python#19. The branch stays. |
tae898
added a commit
to ArcadeData/arcadedb-python
that referenced
this pull request
Nov 3, 2026
…gin reply, and the unwind reads no state it already has (#201) Port of humemai/arcadedb-embedded-python#522 (branch perf/native-tx-crossings, never merged there) onto current main; the P-03 guard of that PR's base is already here. run_in_transaction made 4 library crossings on the happy path (mark, begin, commit mark, commit) although the begin reply already carries {"active","depth","commits"}: the witness now comes from that reply, 3 crossings (the mark must stay before begin). The unwind after a failed block asked is_transaction_active() before every rollback and once more at the end, and the retry check asked for the commit count again; every rollback reply carries the same state, so the unwind returns the state of its last crossing and the retry check reads the commit count from it (a failed crossing falls back to a read). Failed block 6 -> 4 crossings, refused and retried attempt 11 -> 7. begin() and rollback() stay public and return None; _begin_state() and _rollback_state() are the seams run_in_transaction calls. test_interrupt_hygiene.py patched begin and rollback, which the native run_in_transaction no longer calls, so it patches the seam the active backend calls. test_native_tx_crossings.py counts the arcadedb_n_tx operations and pins the behaviour the marks exist for. Counts only, no speed claim: a timed A/B decides it before merge. Refs #19 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backlog row N-05 (bindings study 2026-10-25).
NativeDatabase.run_in_transactionmade 4 library crossings on the happy path (_transaction_stack_mark,begin,_transaction_commit_mark,commit), each one a ctypes call, a malloc'd JSON reply,string_at, a decode and ajson.loads, although thebeginreply already carries{"active","depth","commits"}(ArcadeNative.txreturnstxState(db)for every op) andbegin()discarded it.What changed (Python only,
_native/database.py; no Java change, no image rebuild):run_in_transactiontakescommits_at_startfrom thebeginreply (new internal_begin_state(), publicbegin()still returns None). The unwind after a failed block used to askis_transaction_active()before every rollback and once more at the end, and the retry check asked for the commit count again; every rollback reply carries the same state, so the new_unwind_opened_since(mark)returns the state of its last crossing and the retry check reads the commit count from it. A crossing that failed leaves the state unknown and the next step reads it again, as before._end_transactions_opened_since(mark)keeps its signature and behaviour (it is whatTransactionContextcalls, shared with the JVM wheel).What was measured, counts only (
lib.callinvocations, warmed database, new test filetest_native_tx_crossings.py):run_in_transactionhappy path 4 -> 3 (mark, begin, commit); a block that raises 6 -> 4 (mark, begin, state read, rollback); a refused attempt that is retried and then commits 11 -> 7;with db.transaction()stays at 3 (mark, begin, commit; it never read the commit count) and its failing form is 3 (mark, begin, rollback). Two crossings is not reachable from Python alone: the mark has to be taken beforebegin(a mark taken after it counts the levelbeginpushed as the caller's, ArcadeData#9970/ArcadeData#9321, and the interrupt-between-push-and-return window needs the pre-begin state), so it is a crossing of its own. A Java-side record of the pre-begin state in thebeginop would make it 2; that needs an image rebuild and is not part of this PR.No speed claim is made. Whether 1 of 4 crossings is visible in a small-transaction rate would be a timed A/B through
tools/ab.py, not run here.Semantics: unchanged by design, pinned by tests that pass before and after (a nested block that fails leaves the caller's transaction open, one that succeeds commits with the caller, a refused attempt is rolled back before the retry, a duplicated key is retried once, an error without a retry kind is raised after one attempt). The tests that count crossings fail on
native-preview(3 failures) and pass here. Ran with the 26.10.2.dev1 library and this checkout's Python (test_native_tx_crossings.py,test_transaction_context_interrupt.py,test_core.py,test_concurrency.py,test_run_in_transaction_retry.py: 62 passed, the JPype-only cases of the last file are skipped as before).The P-03 guard bug found while reading is fixed in #527, which is based on
native-previewand goes first. This branch is rebased onto #527's branch, so its first commit is that fix and the retry check here keeps it (the witness is read from the unwind state, the comparison and theraisesit outside thetry). Merge #527 first and this PR shows only its own commit.Repaired in this PR:
tests/test_interrupt_hygiene.pypatchedDatabase.beginandDatabase.rollback, andrun_in_transactionno longer calls either on the native Database, so the patches never applied. Two tests failed (test_a_begin_that_refuses_leaves_the_callers_transaction_alone,test_a_failing_rollback_does_not_skip_the_levels_below_it) and three interrupt tests failed withDID NOT RAISE KeyboardInterrupt, whichconftest.pyturns into a silentNA-JVMskip.rollback()now wraps a new_rollback_state(), asbegin()wraps_begin_state(), and the tests patch the seam the active backend'srun_in_transactionreally calls (begin/rollbackon the JPype Database,_begin_state/_rollback_stateon the native one). They check what they checked before; with the unwind changed to stop at its first failing rollback, the rollback test fails. The one test with no native equivalent (an interrupt betweenbeginand a separate commit-mark read, which the begin reply made one crossing) skips with a stated reason instead of by marker.Local checks on this head: native (
nnextlibrary)test_interrupt_hygiene.py,test_run_in_transaction_partial_commit.py,test_native_tx_crossings.py,test_core.py,test_run_in_transaction_retry.pyandtest_concurrency.py: 77 passed, 10 skipped (nine JPype-only cases of the retry file, as before, and the one test above); JVM wheelwheel-main-78e97b1cff(main), the hygiene and partial-commit files: 20 passed. black, isort and the other pre-commit hooks on the touched files: clean; bandit on_native/database.py: no findings.🤖 Generated with Claude Code