Repository navigation
Description
The two wheels are meant to return the same values. The test suite runs the same tests/ on both, but there is no cross-wheel comparison and no golden file, so a difference stays invisible until a user trips over it. A read-only audit of the tag 26.11.1 (commit 2f28a328be, nothing run) found the following. What is read from the code is marked read, what depends on library behaviour I did not read is marked guess.
Confirmed from the code: to_json_list() returns different Python floats for the same FLOAT or float32 vector value. The native wheel returns the exact float64 widening of the float32 (NativeCodec.java:149-154, 243-261, codec.py:456-480). The JVM wheel passes each float to the engine's JSONObject as a JsonPrimitive (read), and I expect the JSON writer to print the shortest float text such as 0.1 that Python parses as a different double (guess, the writer was not read). It affects scalar FLOAT properties and float[] of any length, not only vectors. On the JVM wheel to_json_list() and to_list() of the same rows disagree, row["vec"] == arr.tolist() is false there and true on native, and a hash or json.dumps of the rows differs. np.allclose and recall are not affected. Only a native-only test pins this (tests/test_native_json_list.py:223-236).
Likely differences, each a guess until a test shows it: to_json_list of DECIMAL (the JVM wheel may give an exact int or number where native gives float(Decimal)); the text of Result.to_json() (native rebuilds it in Python, so key order, separators and float digits may differ); float-list columns in to_columns, to_dataframe and to_arrow (native guesses a float32 array when every value survives float32, the JVM wheel may give a list column); and the last bit of vector distances (the JVM varies between calls, native is repeatable).
Plan, in this order. First one shared test with a fixed table and a committed golden literal, run on both wheels, which shows each difference as a failing assertion (the audit lists 14 cases, starting with the four above). Then fix what the tests show: the float widening in the JVM wheel's RowBatcher.keepNonFinite is a bridge-Java change that fits a wheels-only 26.11.1.post1, after measuring the transport time because the text gets longer. This is part of the work to make both wheels read and write big data well; I will not start before the paper deadline.
Activity
JVM to_json_list: a FLOAT and a float32 float[] arrive as the exact d…
JVM to_json_list: a FLOAT and a float32 float[] arrive as the exact d…
tae898 commentedon Nov 14, 2026
Status. The float half is fixed: #320 makes the JVM wheel widen a
FLOATand a float32float[]to the exact double into_json_list, and addstests/test_cross_wheel_golden.py, one fixed table with committed golden values that runs unchanged on both wheels (floats,DECIMAL, aLONGbeyond 2^53,DATE,DATETIME,NULL, an embedded list). It is in26.11.1.post1and26.11.1.post2. The test fails on the old JVM code and passes on the new one. It is a behaviour test; no speed was measured.Not done: the golden test does not yet cover the text of
Result.to_json(), float-list columns into_arrow, vector distances, or an integralDECIMALinto_json_list. And I have not measured what the longer float text costs the JVM wheel (a float32 now prints about 17 digits instead of about 8); on26.11.1the JVM wheel was already the slower one onto_json_listanditer_json_batchesof the 128-dimension table (#312). I will measure that on the26.11.1.post2pair. The issue stays open for those two items.