[tae898]

For users nothing changes in the values or the errors: on the native wheel insert_columns(..., parallel=True) now sends the columns to the library in one call instead of turning them into one dict per row and one JSON text per 10,000 rows, and the same records are stored. Closes #325 (part of #315). Base is release/26.11.1, for the wheels-only 26.11.1.post1.

What changes: the columnar entry arcadedb_n_insert_colp takes "parallel": true in its header. The library builds each record from the column buffers (the same setRow code as the synchronous call) and hands it to db.async().createRecord with a failure token, as DocumentBatcher.insertColumnsParallel does on the JVM wheel. No transaction is opened, committed or rolled back there and commit_every does not apply, which is what the parallel row path already did. The Python side makes the failure token before the crossing (a new token operation of arcadedb_n_async) and passes its id in the header ("fid"), then waits for the writers and reads the failures through the same code as insert_many(parallel=True) (_finish_parallel, split out of _insert_many_parallel), so the error text, the "rows may have been stored" rule and the failed-record count are the same. A column set the buffers cannot carry (mixed types, nested values, date-times) still takes the row path, and ARCADEDB_NATIVE_BULK=0 still turns the columnar path off for parallel calls too. An edge type is refused with the same message as before.

insert_arrow and insert_dataframe with parallel=True on a document type had the same fall back (the columns became Python values and went to insert_many(parallel=True)), so they now keep their columns for the same entry (dataframe.py, _insert_documents_native); a batch the buffers cannot carry still takes the row path. Vertex types still refuse parallel=True there, as before.

This is Java glue: the native library is rebuilt from the unchanged engine commit, no engine file is touched. The two glue files are BulkOps.java (the parallel branch, one setRow shared by both modes) and NativeExtras.java (the failure token that insert_parallel already used, now a small class used by both, with the token operation that makes one). Memory: the call no longer builds the per-row dict list and the JSON texts, and the records are handed over as they are built (the same hand-over as the JVM wheel's parallel call); I did not measure the peak.

Tests (native only, tests/test_native_columnar_parallel.py): a spy proves that a parallel call makes one arcadedb_n_insert_colp crossing and no insert_parallel chunk (it fails on the old code, which made the row call); the stored records, with value and Python type per property, equal those of insert_many(parallel=True) with dict rows for a document type, a vertex type and a column set with nulls and non-ASCII text; a duplicate key and a value the declared type refuses are reported after the wait; edge type, missing type, mixed-type fallback, the off switch, empty columns and the unchanged synchronous call are covered. The golden test tests/test_cross_wheel_golden.py is not touched.

No speed claim is made here. The measurement is the untimed profile study of the ingest shapes and a later laptop A/B on this change. I could not run any test locally (CI only); the CI runs are linked in the first comment.

Changes after an independent review of this PR, each one a behaviour change worth reading. An interrupt (KeyboardInterrupt, any BaseException) during the crossing, or between the chunks of insert_many(parallel=True), now waits for the writers and reads (so removes) the failure tokens before it propagates, as the row path already waited; before, the columnar path caught only ArcadeDBError and left the writers running with no wait. Because the caller owns the token and sends its id in the header, an interrupt that lands after the records are handed over cannot lose the id. _finish_parallel now reads every token even when the wait itself raises; before, the failure counters of an unread token (ASYNC_FAILS and ASYNC_FIRST, which hold a Throwable) lived for the life of the process, and the row path of insert_many(parallel=True) had the same leak, which is fixed there too. A failure while the image hands records over now says how many it had handed over ("after handing N of M rows to the async writers (those may have been stored)") and the Python message adds the failed records the writers had already counted; Java no longer drops the token, so those counts are not discarded. insert_table(parallel=True) inside a caller's transaction no longer says the earlier batches were "written in your transaction, not committed" (a pre-existing message that was false for async writes): the writers commit on their own, a rollback of the caller's transaction does not remove those rows, and the message now says that.

Review tests (native only, tests/test_native_columnar_parallel_review_fixes.py): a duplicate-key load asserts stored + reported failed >= n (the count is parsed from the message) and stored == number of distinct ids; a load of threads * 10240 + 1 rows, one past a full batch for every writer, stores every row and the count is unchanged after the call returns; an interrupt in the crossing (the real call runs, then KeyboardInterrupt) leaves is_processing() false and the token read, and so does one between the chunks of the row path; a wait that raises still reads the tokens; the in-transaction message does not promise a rollback and the committed rows survive one.

In parallel mode a duplicate key abandons the open batch of the failing writer: the async writers roll back their whole open batch (up to commit_every rows) when one record fails, so the valid rows of that batch are not stored and are reported in the error with the failed one. A parallel load therefore guarantees stored + reported failed >= rows, no row stored twice and only sent rows stored, not that every distinct key is stored; the review test asserts exactly that (the first CI run showed 1,000 of 2,950 distinct keys stored for a load with 50 repeated keys).

🤖 Generated with Claude Code

[tae898]

CI evidence for 25bceaa8d, dispatched on this branch (a draft PR skips the platform legs, and the PR runs use the latest jars): Test native version=26.11.1.post1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42597223330, Test JVM arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42597226741, Test Python Examples arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42597230427. Results will be added here.

🤖 Generated with Claude Code

[tae898]

I applied an independent review to this PR (new head c24f154). All four findings were right and are fixed: the columnar parallel path now waits for the writers and reads the failure token on any BaseException (an interrupt included), _finish_parallel reads every token even when the wait raises (the row path leaked the same entries, fixed there too), the failure text says how many rows were handed over, and the in-transaction message of insert_table(parallel=True) no longer claims the rows are uncommitted. The token is now made by the caller and its id travels in the header, so the Java side never drops it. Tests for each are in tests/test_native_columnar_parallel_review_fixes.py; the PR body lists the behaviour changes. I could not run anything locally, so CI is the evidence: native, jvm, examples on the fork.