[tae898]

What does this PR do?

On the JVM wheel, insert_many turned every chunk of 10,000 rows into one JSON text (json_bulk_dumps_tagged), which DocumentBatcher.insertManyJson parsed again into a JSON tree and copied key by key. Now a chunk whose rows are dicts with the same str keys in the same order, and whose columns each hold None and values of one exact type (int within 64 bits, float, bool, str, date, or a datetime without a time zone), crosses as typed columns and no text: one numpy buffer per int, float or bool column, one StringColumn per str column (what insert_columns already sends), one TemporalColumn per date or datetime column (what #383 added), and a boxed Object[] only for a numeric column that holds a null. The new bridge entry points are DocumentBatcher.insertManyColumns and insertManyColumnsParallel, which share fillRow with insertColumns. The synchronous and the parallel mode both take them.

Every other chunk keeps the JSON text and the per-row fallback exactly as it was. The chunk is judged as a whole, from the types alone, before any bridge array is built: a row that lacks a key, has another one or lists the keys in another order, a column that mixes types (an int beside a float), an int beyond 64 bits, a Decimal, bytes, numpy or pandas value, a datetime with a time zone, a lone surrogate, a list, tuple or dict (a date inside a list included), and a float vector for an ARRAY_OF_FLOATS property. A fallback chunk pays for the census and nothing else. The bridge jar and Python change; no native image is involved.

One decision in the Java side is not obvious. The JSON text stores a whole number as an Integer when it fits an int and as a Long beyond (JSONObject.parseNumber), while a numpy long[] column would store a Long for every number. A schemaless property would then be an Integer or a Long depending on which route carried its chunk, and a load with one fallback chunk would mix the two. So insertManyColumns narrows a whole number the way the text did. insertColumns is unchanged and keeps Long (a test pins it).

None stays a null property, never a missing one. A float NaN, an infinity and a subnormal cross in a double[] and are stored as the per-row path stores them (the old route sent those rows down the per-row path; the parity tests compare the stored values).

What the change allocates per chunk of at most 10,000 rows and per column: one Python list of the column's values (references, not copies); an int, float or bool column also one numpy array and one long[], double[] or boolean[] copy; a str column one UTF-8 byte buffer, one int[] of offsets and a boolean[] when there is a null (the insert_columns layout); a date or datetime column one numpy int64 array, one long[] and a boolean[] when there is a null; a numeric column with a null one Object[] of boxed values. The chunk is no longer held as one JSON text in Python, as a Java String copy of it, and as a parsed JSON tree of all its rows. The memory of a load stays bounded by the chunk, as before (#294). I have not measured memory.

Related issues

Refs #388 and #315.

Performance changes

Not measured yet. The change removes the text and the parse, and I have not timed it. The speed claim waits for a pre-registered timed A/B of a candidate wheel against 26.11.1.post3.

The figure to measure, so the A/B can be registered before it runs:

• Operation: Database.insert_many(type_name, rows) on the JVM wheel (arcadedb-jvm), a list of dicts, default commit_every, synchronous mode. The parallel mode as a second figure.
• Sizes: 1, 100, 10,000 (one chunk) and 100,000 rows (ten chunks). The unit of one operation is one insert_many call; report rows per second and the latency per call.
• Shapes: a narrow row (an int, a float, a str), a wide row of nine typed properties with date and datetime columns (the shape of the insert_columns figure), and the same wide row with a null in some columns of every chunk.
• Memory: peak RSS and anon growth of the load, as information figures.
• No-change control: a shape the typed route refuses with the same row count, for example one row per chunk with an extra key, or a Decimal column. Both arms run the old route there, so the ratio should be one. An A/A run of the released wheel as the noise control.
• Parity digest: the stored data of the old route and the new route, read back through SQL SELECT and compared property by property (names in the order the record lists them, the Java class of each value, the value), for each shape. tests/test_insert_many_typed_columns.py does this for ints around 2^31, 2^53 and 2^63, floats (NaN, infinities, subnormal, -0.0), strings, dates, datetimes, nulls, None against a missing key, key order, mixed columns, a date or Decimal inside a list, bytes, numpy scalars, a datetime with a time zone, vertices, an edge type, a duplicate key and a value no route can store.

Docs

docs/modules/ROOT/pages/how-to/bulk-loads.adoc (the section on the values the bulk path carries) says which chunks cross as columns on the JVM package and which keep the JSON text. The insert_many docstring says the same. README.md does not describe the transport.

Checklist

• A regression test fails on the old code and passes on the new (the first commit holds the tests, the second the change; the dispatched run of the first commit is the evidence, see the comment below)
• The test suite that the change can affect passes
• uvx pre-commit run --files <changed files> passes (Bandit as CI runs it is clean too)

🤖 Generated with Claude Code