[tae898]

For users the stored vertices are the same: insert_dataframe and insert_arrow on a vertex type now load through the columnar insert on both wheels instead of GraphBatch.create_vertices with one dict per row and a JSON text per chunk. Three things differ and I want them read before merging: a vertex load now runs with the database's write-ahead log settings (the old route turned the log off for the load, the GraphBatch default), it commits once per batch of batch_size rows (default 10,000; the old route committed per 1,000 rows on the native wheel and per 100,000 on the JVM wheel), and the commit retry that GraphBatch.createVertices has is now done by the chunk loop of the columnar insert itself on both wheels (below). Part of #319. Base is release/26.11.1, for the wheels-only 26.11.1.post1.

This PR is stacked on #333 (it needs the Arrow columns that PR keeps for the entry, in dataframe.py) and #328, so its diff shows their commits until they merge; the change of this PR is the last commit. Merge #328, then #333, then this.

What I checked in the code before routing (engine at the 26.11.1 commit a3a5d12457, GraphBatch.java): createVertices(type, properties) is database.newVertex(type), set per property, save(), inside one transaction that it begins and commits, on the caller's thread. BulkOps.insertColumns (native) and DocumentBatcher.insertColumns (JVM) run the same loop for a vertex type (db.newVertex, set, save, then the batch commit). The bucket of a vertex is chosen by the type's bucket selection strategy inside save() and the index entries are written by the engine at save() and commit, so the buckets and the indexes are the same by construction; the test compares them anyway (per-bucket counts of a 3-bucket type, WHERE lookups on a unique and a non-unique index, lookup_by_key). The RIDs were discarded by insert_table already, and a GraphBatch that holds no edges has nothing else to do (its known-new-vertex set only serves the edge pass). The differences the code shows: the GraphBatch builder defaults useWAL to false, so the old route relaxed durability for the load and restored it at close, while the columnar insert uses the database's settings, as the document loads of the same calls always did (this can make a vertex load slower or not, I did not measure it, and that A/B decides whether this PR ships); the batch also held the speculative background maintenance of the indexes (the vector index compaction) for its lifetime, which the columnar insert does not; and createVerticesWithRetry retried a commit that failed with a retryable error (an MVCC conflict, a lock timeout, a Raft re-election), which the chunk loop of the columnar insert does now (below).

The rule that a vertex load refuses to run inside a transaction the caller opened comes from GraphBatch.createVertices (#9242, requireNoCallerTransaction): the engine throws before touching anything and the Python wrapper raised it as an ArcadeDBError. The columnar insert would join the caller's transaction instead (as a document load does), so insert_table now raises the refusal itself, before the first batch, with the engine's wording ("cannot run inside a transaction the caller opened"), and writes nothing. A document load still joins the caller's transaction. parallel=True on a vertex type is still refused with the same message as before.

The failure message is now exact on the native wheel too: a duplicate key rolls back the whole batch of batch_size rows (before, the native wheel had committed the chunks of 1,000 rows before the failing one), and earlier batches stay, which is what "the N row(s) of the batches before this one were committed" says. Not changed: graph_batch().create_vertices itself, the insert_many route and the docstring of Database.insert_arrow in core.py (it still says a vertex type loads through GraphBatch; I left that file to the PR that owns it, the line needs one word changed).

Tests: tests/test_vertex_load_columnar.py runs on both wheels: it loads a table of vertices through graph_batch().create_vertices (the private reference of the old route) and through insert_dataframe and insert_arrow, with create_vertices patched to fail so the old code fails this test, and compares every stored property with its Python type (nulls, non-ASCII text, a pre-1970 nanosecond timestamp, int32 and float columns, a float32 vector), the per-bucket record counts and the index lookups; it also covers the duplicate-key rollback, the refusal inside a caller transaction (and the document load joining it). tests/test_native_vertex_columnar.py (native only) spies on the crossings: one arcadedb_n_insert_colp per batch, no JSON text, no graph batch call. The golden test tests/test_cross_wheel_golden.py is not touched.

Memory: the load no longer builds a dict per row and a JSON text per chunk. A later laptop study (cores 12-15, 3 clean cells per arm) found one more copy: an Arrow vector column was turned into Python floats for every batch (22.9 MiB of Python objects for 10,000 rows of 64 dimensions, 34.8 MiB for 96), alive for the whole crossing. It now reaches the columnar entry as the Arrow array (commit 447a950): the Python-side peak of those calls is 0.0 MiB and they run about 2 times faster. The anonymous memory of the process after the load (RssAnon, 281 MiB against 239 MiB for the old route on the 50,000-row product load) is the image's heap, not Python, and this change does not move it. No speed claim is made here: the measurement is the untimed profile study of the ingest shapes and a later laptop A/B on this change, with the WAL difference above in mind. I could not run any test locally (CI only); the CI runs are linked in the first comment.

Behaviour changes in this PR, all of them stated here. The write-ahead log is on for a vertex load, as the database is configured: the old route turned it off for the load (the GraphBatch useWAL default) and restored it at close, so this is no longer an async-flush relaxation, and neither is the hold the GraphBatch kept on the speculative index maintenance for its lifetime. A vertex load commits once per batch_size rows (default 10,000) instead of per 1,000 rows (native) or 100,000 rows (JVM). The text of a vertex failure changed from "Failed to create batch vertices for type ..." to "Failed to bulk-insert into '': ..."; only the refusal inside a caller's transaction keeps the old wording.

Changes after an independent review: the retry. The old route rolled a vertex chunk back and wrote it again on any NeedRetryException (the engine's commitRetries default is 10, the delay starts at 1 s and doubles to at most 10 s). The new route was one plain transaction per chunk, so two threads loading one vertex type with a non-unique index could get a commit conflict as an error with earlier chunks already committed. BulkOps.insertColumns (native wheel) and DocumentBatcher.insertColumns and insertManyJson (JVM wheel) now write each chunk in a transaction of their own, roll it back on a NeedRetryException and write it again from its first row, with the engine's bound, and throw the error unchanged after the last attempt. Documents benefit too. Inside a transaction the caller opened nothing is retried: the rows join it, as before. Not covered: on the native wheel the JSON row path (insert_many, which commits in Python) has no retry; a vertex or document batch reaches it only when a column cannot cross as memory (a list of strings, for example), as it did for documents before this PR.

Review tests (both wheels, tests/test_vertex_load_review_fixes.py): 2 and 4 threads load one vertex type with a non-unique index on five key values, 20 batches each, and the count equals the total, with no row written twice; a duplicate key in the third of five batches leaves the first two committed, leaves no row and no index entry of its own (lookup_by_key returns none for its keys), states the committed count in the message, and the fixed tail loads afterwards.

🤖 Generated with Claude Code

[tae898]

CI evidence for abed8dfd7, dispatched on this branch (a draft PR skips the platform legs, and the PR runs use the latest jars): Test native version=26.11.1.post1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598698068, Test JVM arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598701874, Test Python Examples arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598705349. Results will be added here.

🤖 Generated with Claude Code

[tae898]

I applied an independent review to this PR (new head 3a899ae, rebased on the updated #333). A is fixed on both wheels: the chunk loop of BulkOps.insertColumns (native) and DocumentBatcher.insertColumns and insertManyJson (JVM) rolls a chunk back on a NeedRetryException and writes it again from its first row, bounded like GraphBatch (10 retries, 1 s back-off doubling to 10 s), and never retries inside a caller's transaction. B and C: the message change is stated in the body, and the test for a duplicate in the third of five batches is added, together with a 2 and 4 thread test on one vertex type (tests/test_vertex_load_review_fixes.py). Not fixed: the native JSON row path (insert_many, which commits in Python) still has no retry; a vertex batch reaches it only when a column cannot cross as memory. I could not run anything locally, so CI is the evidence: native, jvm, examples on the fork.