Skip to content

Parity: the two wheels return different Python floats from to_json_list, and no cross-wheel golden test exists #314

Open

Description

@tae898

The two wheels are meant to return the same values. The test suite runs the same tests/ on both, but there is no cross-wheel comparison and no golden file, so a difference stays invisible until a user trips over it. A read-only audit of the tag 26.11.1 (commit 2f28a328be, nothing run) found the following. What is read from the code is marked read, what depends on library behaviour I did not read is marked guess.

Confirmed from the code: to_json_list() returns different Python floats for the same FLOAT or float32 vector value. The native wheel returns the exact float64 widening of the float32 (NativeCodec.java:149-154, 243-261, codec.py:456-480). The JVM wheel passes each float to the engine's JSONObject as a JsonPrimitive (read), and I expect the JSON writer to print the shortest float text such as 0.1 that Python parses as a different double (guess, the writer was not read). It affects scalar FLOAT properties and float[] of any length, not only vectors. On the JVM wheel to_json_list() and to_list() of the same rows disagree, row["vec"] == arr.tolist() is false there and true on native, and a hash or json.dumps of the rows differs. np.allclose and recall are not affected. Only a native-only test pins this (tests/test_native_json_list.py:223-236).

Likely differences, each a guess until a test shows it: to_json_list of DECIMAL (the JVM wheel may give an exact int or number where native gives float(Decimal)); the text of Result.to_json() (native rebuilds it in Python, so key order, separators and float digits may differ); float-list columns in to_columns, to_dataframe and to_arrow (native guesses a float32 array when every value survives float32, the JVM wheel may give a list column); and the last bit of vector distances (the JVM varies between calls, native is repeatable).

Plan, in this order. First one shared test with a fixed table and a committed golden literal, run on both wheels, which shows each difference as a failing assertion (the audit lists 14 cases, starting with the four above). Then fix what the tests show: the float widening in the JVM wheel's RowBatcher.keepNonFinite is a bridge-Java change that fits a wheels-only 26.11.1.post1, after measuring the transport time because the text gets longer. This is part of the work to make both wheels read and write big data well; I will not start before the paper deadline.

Activity

  1. added
    jvmThe arcadedb-jvm package (JPype build)
    nativeThe arcadedb package (native library build)
    pythonPull requests that update python code
    vectorVector indexes and find_nearest
    on Nov 9, 2026
  2. added this to the 26.12.1 milestone on Nov 9, 2026
  3. self-assigned this
    on Nov 9, 2026
  4. tae898 commented on Nov 14, 2026

    @tae898
    CollaboratorAuthor

    Status. The float half is fixed: #320 makes the JVM wheel widen a FLOAT and a float32 float[] to the exact double in to_json_list, and adds tests/test_cross_wheel_golden.py, one fixed table with committed golden values that runs unchanged on both wheels (floats, DECIMAL, a LONG beyond 2^53, DATE, DATETIME, NULL, an embedded list). It is in 26.11.1.post1 and 26.11.1.post2. The test fails on the old JVM code and passes on the new one. It is a behaviour test; no speed was measured.

    Not done: the golden test does not yet cover the text of Result.to_json(), float-list columns in to_arrow, vector distances, or an integral DECIMAL in to_json_list. And I have not measured what the longer float text costs the JVM wheel (a float32 now prints about 17 digits instead of about 8); on 26.11.1 the JVM wheel was already the slower one on to_json_list and iter_json_batches of the 128-dimension table (#312). I will measure that on the 26.11.1.post2 pair. The issue stays open for those two items.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

jvmThe arcadedb-jvm package (JPype build)nativeThe arcadedb package (native library build)pythonPull requests that update python codevectorVector indexes and find_nearest

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions