[tae898]

For users nothing changes in the stored values: on the native wheel a batch with a date-time, date, Decimal or bytes column now stays on the columnar entry instead of turning every column of the batch into a dict per row and a JSON text with tagged values, and the same objects are stored. Part of #316 (all four kinds are in; I could prove each conversion from the code, see below). Base is release/26.11.1, for the wheels-only 26.11.1.post1.

This PR is stacked on #328 (it needs the same BulkOps.java and dataframe.py code, and both change the value switch), so its diff shows that PR's two commits until it merges; the change of this PR is the last commit. Merge #328 first.

What changes: BulkOps.insertColumns gets four kinds, each read as the object that Json.decodeTag builds for the tagged value the row path sends. T is a date-time: int64 ticks of the UTC wall clock since 1970-01-01T00:00:00 with dim ticks per second (1, 1000, 10^6 or 10^9, so an Arrow timestamp keeps its unit and a nanosecond value keeps its nanoseconds, as _temporal_to_tagged did), read with floorDiv and floorMod as a LocalDateTime (the object LocalDateTime.parse of the tag gave, so pre-1970 values are exact). D is a date: int32 epoch days, a LocalDate. n is a decimal: str(Decimal) in the string layout read with new BigDecimal(text), which is what the n tag does with the same text, so the scale is kept (1E+3 stays 1E+3). x is bytes in the string layout read as the byte[] the b tag decodes to. columnar.py lays out lists of datetime (also pandas.Timestamp, microseconds as _tag_datetime keeps them), date, Decimal, bytes and bytearray, and Arrow timestamp, date32, date64, decimal and binary arrays. The time zone rule is not changed: a time-zone-aware value is its UTC wall clock and a naive one is taken as it is, the rule of _tag_datetime, and nothing in the column decides a zone. Still refused (row path as before): numpy datetime64 arrays and elements, a column that mixes types (a datetime and a date, bytes and text, bytes and bytearray), a non-finite Decimal, and pandas NaT and NA columns. insert_arrow and insert_dataframe keep those Arrow columns for the entry and turn them into tagged values only when a batch falls back to the row path (another column, such as a list of strings, cannot cross), so a fall back stores what it stored before.

This is Java glue: the native library is rebuilt from the unchanged engine commit, no engine file is touched. Memory: the batch no longer builds the per-row dicts and the JSON text, but a list of Python date-times is converted to an int64 array first (one 8-byte value per row); I did not measure the peak.

Tests: tests/test_native_columnar_temporal.py (native only) loads the same rows and Arrow tables through the row path (the columnar path switched off, the private reference of the old behaviour) and through the columns, compares every stored value and its Python type, and spies on the crossings (the old code makes no arcadedb_n_insert_colp call there, so these fail on it); it covers declared and undeclared properties, nulls, pre-1970 values, aware and naive date-times, pandas Timestamps, Arrow units s, ms, us, ns with and without a zone, date32, decimal128, binary, the fall back with a list column, a parallel load, and the image's refusal of a wrong tick unit. tests/test_columnar_temporal_lists.py runs on both wheels with literal expected values (naive, aware to UTC, pre-1970, null, date, Decimal, bytes), so the same stored values on the JVM wheel and the native wheel are a CI result; the existing tests/test_dataframe_ingest.py precision, zone and date tests run through the new path on the native wheel. tests/test_native_columnar_pack.py pins the layouts without a library, and three existing tests that asserted the old fall back (a date-time column, a Timestamp column, an Arrow timestamp) now assert the columnar entry. The golden test tests/test_cross_wheel_golden.py is not touched.

No speed claim is made here. The measurement is the untimed profile study of the ingest shapes and a later laptop A/B on this change. I could not run any test locally (CI only); the CI runs are linked in the first comment.

Changes after an independent review of this PR. A date64 value beyond int32 days no longer wraps in the int32 cast (it did, silently; the row path raised): columnar.pack now returns None for it, so the batch takes the row path and raises what it raised before, with nothing stored. A timestamp[s] beyond the years of java.time (about 3.15e16 seconds) made LocalDateTime.ofEpochSecond throw in the middle of a batch; pack now checks the range in Python and returns None, so the row path raises its usual error before anything is stored (the synchronous call leaves no transaction open; a parallel call says that rows may have been stored, as it can have handed rows over before the failure). The review named timestamp[s] and timestamp[ms]: only s can pass the range, because the largest int64 of milliseconds is 9.2e15 seconds against 3.15e16, so milliseconds, microseconds and nanoseconds need no check (a test loads the int64 limits in those units and expects them to stay columnar).

One behaviour change to read: an insert_many chunk that mixes a date-time, date, Decimal or bytes column with a float column holding NaN now goes columnar, and the NaN is stored as Double.NaN; before, the temporal column forced the JSON path, where a NaN row goes through INSERT INTO t CONTENT :row on its own. I read the engine at a3a5d12457: CONTENT is UpdateContentStep, which calls doc.fromMap, and fromMap and doc.set both end in MutableDocument.putProperty with no special case for NaN, so the stored value is the same Double.NaN on both routes (an INSERT of a NaN into a declared DOUBLE keeps it). That is read from code; tests/test_native_columnar_temporal_review_fixes.py compares both routes by value and type, and the CI run is the proof.

Review tests (native only, tests/test_native_columnar_temporal_review_fixes.py): the pack-level limits of date64 and timestamp[s] (one value past the limit is enough to refuse the column, the limit itself stays columnar and exact); pa.array([0, 2**62], pa.timestamp("s")) and a date64 past int32 days through the synchronous call (nothing stored, no transaction left open) and the parallel call (may have been stored); and the NaN comparison above.

A chunk of insert_many rows in which one row holds a value the JSON text cannot carry still sends that row to the statement on its own, and the runs of rows around it stay on the bulk entry; since the columnar entry now carries date-time, date, Decimal and bytes columns, such a run crosses on arcadedb_n_insert_colp where it used to cross on the JSON entry (arcadedb_n_insert_rows). A Decimal column that holds a non-finite value (NaN, sNaN, Infinity) is refused as a whole, in lists, with nulls and in object arrays (new BigDecimal("NaN") would throw in the middle of a batch), so the non-finite row takes the statement path as before.

🤖 Generated with Claude Code

[tae898]

CI evidence for 4546c437b, dispatched on this branch (a draft PR skips the platform legs, and the PR runs use the latest jars): Test native version=26.11.1.post1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598022837, Test JVM arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598026486, Test Python Examples arcadedb-tag=26.11.1 https://github.com/ArcadeData/arcadedb-python/actions/runs/42598030115. Results will be added here.

🤖 Generated with Claude Code

[tae898]

I applied an independent review to this PR (new head b1001e2, rebased on the updated #328). A and B are fixed in Python before the crossing: a date64 past int32 days and a timestamp[s] past the java.time range make pack return None, so the row path raises as before with nothing stored. One correction to the finding: it named timestamp[s] and timestamp[ms], but only s can exceed the range (an int64 of milliseconds is at most 9.2e15 seconds against 3.15e16), so ms, us and ns are not checked and a test keeps their int64 limits columnar. C is stated in the PR body: from the engine source (CONTENT goes through fromMap, which ends in the same putProperty as doc.set) the stored NaN is the same on both routes; that is read from code and a test compares them. CI on the fork: native, jvm, examples.