Skip to content

JVM to_json_list: a FLOAT and a float32 float[] arrive as the exact double, as on the native wheel (refs #314) - #320

Merged
tae898 merged 1 commit into
mainfrom
fix/314-json-float-parity
Nov 9, 2026
Merged

tae898 merged 1 commit into
mainfrom
fix/314-json-float-parity

Conversation

@tae898

@tae898 tae898 commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

I found that to_json_list() returns different Python floats on the two wheels for the same FLOAT or float32 float[] value. The native wheel returns the exact float64 widening of the float32 (NativeCodec.java widens a Float with doubleValue(), and the float32 tail of a vector is decoded with .tolist()). The JVM wheel passed each Float and each float[] to the engine's JSONObject unchanged (RowBatcher.keepNonFinite returned them as they were), and the writer prints the shortest float32 text, 0.1, which Python reads as a different double than 0.10000000149011612. The JVM wheel's own to_list() widens exactly, so on the JVM wheel to_json_list() and to_list() of the same rows disagreed, and row["vec"] == original.tolist() was false there and true on native.

The change is in RowBatcher.keepNonFinite only: a Float becomes a Double and a float[] becomes a double[] before the writer sees it, and NaN and the infinities keep their tag. The native wheel is not touched. The bridge jar is rebuilt by CI, so the new behaviour is what the JVM test jobs run.

The test is tests/test_cross_wheel_golden.py: one fixed two-row table with committed golden literals for to_list(), to_json_list() and to_columns() (FLOAT, DOUBLE, float32 float[] of 3 and of 9 values, DECIMAL, LONG beyond 2^53, DATE and DATETIME including a pre-epoch value, NULL, an embedded list). It has no branch on the backend, so the same file runs on both wheels and a literal that holds on both is the parity check. The float cases fail on the JVM wheel before the Java change. No case is marked xfail.

The JSON text of a float32 gets longer on the JVM wheel (about 17 digits instead of about 8). I have not measured what that costs in transport time.

Related issues

Refs #314. The xfail list of this PR is empty, but the audit in #314 lists cases this test does not cover yet (an integral DECIMAL in to_json_list, the text of Result.to_json(), float-list columns in to_columns and to_arrow, vector distances), so I do not close it here.

Performance changes

None claimed. The longer JSON text is not measured.

Docs

None needed: the documented contract is that both wheels return the same values.

Checklist

  • A regression test fails on the old code and passes on the new
  • The test suite that the change can affect passes (CI is the first run)
  • uvx pre-commit run --files <changed files> passes

🤖 Generated with Claude Code

…ouble, as on the native wheel (refs #314)

RowBatcher.keepNonFinite now widens a Float to a Double and a float[] to a double[] before the engine's JSONObject writes
it, so the JSON text is the exact float64 widening of the float32 (0.10000000149011612, not the shortest float32 text 0.1).
NaN and infinity keep their tag.

tests/test_cross_wheel_golden.py is one fixed table with committed golden literals for to_list, to_json_list and
to_columns (FLOAT, DOUBLE, float32 float[] of 3 and 9, DECIMAL, LONG beyond 2**53, DATE, DATETIME, NULL, embedded list),
run unchanged on both wheels.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 551dee9 into main Nov 9, 2026
43 of 61 checks passed
@tae898
tae898 deleted the fix/314-json-float-parity branch November 9, 2026 08:44
@github-actions github-actions Bot added this to the 26.12.1 milestone Nov 9, 2026
abhirajadhikary06 pushed a commit to abhirajadhikary06/arcadedb-python that referenced this pull request Nov 10, 2026
…ouble, as on the native wheel (refs ArcadeData#314) (ArcadeData#320)

RowBatcher.keepNonFinite now widens a Float to a Double and a float[] to a double[] before the engine's JSONObject writes
it, so the JSON text is the exact float64 widening of the float32 (0.10000000149011612, not the shortest float32 text 0.1).
NaN and infinity keep their tag.

tests/test_cross_wheel_golden.py is one fixed table with committed golden literals for to_list, to_json_list and
to_columns (FLOAT, DOUBLE, float32 float[] of 3 and 9, DECIMAL, LONG beyond 2**53, DATE, DATETIME, NULL, embedded list),
run unchanged on both wheels.

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
(cherry picked from commit 551dee9)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
abhirajadhikary06 pushed a commit to abhirajadhikary06/arcadedb-python that referenced this pull request Nov 10, 2026
…Data#320 JVM to_json_list float parity, ArcadeData#321 native columnar read assembly)

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
abhirajadhikary06 pushed a commit to abhirajadhikary06/arcadedb-python that referenced this pull request Nov 10, 2026
The release line (tag 26.11.1.post2 = 1454809, which contains the post1 tag
26.11.1.post1 = dbe1670) branched from the 26.11.1 tag and carries the post1
candidates (the native column frames ArcadeData#331, the other native and JVM load and read changes
of ArcadeData#327 to ArcadeData#339, and the cherry-picks of main's ArcadeData#320 and ArcadeData#321),
the post2 additions (Windows parity harness ArcadeData#295/ArcadeData#347, ArcadeData#340, the JVM Arrow
vector load change) and their test fixes. Main did not have them. The merge
uses a merge commit so that the release tag stays an ancestor of main.

Four files conflicted, each because main and the release line changed the
same lines. Resolutions, each checked against the history of both sides:

- src/arcadedb/_native/columns.py (modify/delete): deleted. Main changed it
  only with ArcadeData#321 (6c6a5a1). The release line holds the same change as the
  cherry-pick 946dfda (identical blobs of columns.py and of
  test_native_columns_reference.py at both commits) and then deleted the
  module with the native column frames (ArcadeData#331, d606cce). Nothing else imports
  it; the merged database.py reads columns through the frames code.
- src/arcadedb/_native/database.py: kept both. Main's facets lines in _fill
  and its facets property (ArcadeData#299, ArcadeData#301), then the release line's
  self._note_vectors(reply) before the return and its _note_vectors method.
- tests/conftest.py: the release side of the native-only block (the
  comment of test_native_columns_reference.py now says what the file
  compares, plus the two new native-only files), and both entries in
  WINDOWS_NOT_APPLICABLE (main's test_dev_engine_pin.py and the release
  line's test_parity_gate_awk.py).
- tests/test_native_columns_reference.py (add/add): the release side whole.
  Main's file tested the deleted module.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant