Repository navigation
Conversation
…ject_hook and without reading its side array (refs ArcadeData#315) to_list() and iter_dicts() of the JVM wheel build a json object_hook for every batch and read the side array of every batch, whether or not the batch holds a tagged value. These tests count the hook factory, the object_hook argument of json.loads and the batch elements that are read, for a result of plain values (no hook, no read of the side array) and for every kind of value the bridge tags (the hook stays). They also compare to_list() and iter_dicts() with the decoding that always builds the hook, over single shapes, a mixed result and batch boundaries. They fail on the current decoder (every batch gets a hook and a read of the side array) and pass once the decoder looks for the tag escape in the batch text first. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s text without an object_hook and without reading its side array (refs ArcadeData#315) Every batch was decoded with json.loads(text, object_hook=_typed_hook(side)) and its side array was read first, also when the batch held only plain values. The hook is a Python call per JSON object and the read of the side array is a JPype operation per batch, which for iter_dicts (one row per crossing) is one per row. TypedRows writes every tag as {"\u0001": [kind, payload]} with the escape spelled out, writes a control character of a string as the same escape, and never writes the raw character, and every side array entry is referred to by a tag in the text. A text without the escape therefore holds no tag and no side index, the hook would return each object unchanged, and plain json.loads gives the same value. The check is a substring scan of a str that already exists (_loads_typed_row did it for one row). A string that holds U+0001 or the text of the escape only keeps the hook, which then changes nothing. No Java change, and the native wheel is untouched. The speed effect is not measured: it waits for a timed A/B of a candidate wheel against 26.11.1.post2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This was referenced Nov 13, 2026
Collaborator
Author
tae898
added a commit
that referenced
this pull request
Nov 14, 2026
…, and the stale statements (refs #315) (#399) JVM iter_dicts parses a batch with no tagged value without the object hook (#382); JVM insert_arrow and insert_dataframe carry a timestamp or date column as one long[] of epoch ticks (#383); native transaction calls return the state as one integer (#384); native save() of a saved record with auto-transaction on skips the transaction check (#385). The figures are copied by script from the scored gate study and each is checked against the public comment on #315. The n_update_one figure ran with auto-transaction off, so the page says off (the comment says on). The save() paragraph said three calls without the auto-transaction case. The quickstart was re-run on both published 26.11.1.post3 packages on Python 3.12 and printed the output shown. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The JVM wheel's
to_list()anditer_dicts()decode everyTypedRowsbatch withjson.loads(text, object_hook=_typed_hook(side))and read the batch's side array first, also when the batch holds only plain values. The hook is a Python call per JSON object, and reading the side array is a JPype operation per batch (per row foriter_dicts(), which reads one row per crossing). I added_loads_typed_batchinsrc/arcadedb/results.py: it looks for the tag escape in the batch text first, and when it is absent it parses with plainjson.loadsand does not touch the side array._loads_typed_rowalready does this for a single row. No Java change, and the native wheel is untouched.How "nothing tagged" is decided, and why that is exact
The side array is not a usable signal: DECIMAL, dates, datetimes and sets are tagged without a side entry. The text is:
TypedRows.javawrites a tag in two places only,tagged()and therrow ofappendRow(), and both spell it with the six characters\u0001.quote()writes a control character inside a string as the same escape, and no path writes the raw character.toSide()and therrow). A text without those six characters therefore has no tag and no side index._typed_hookreturns any object that lacks the tag key unchanged, sojson.loads(text)gives the same value as the hooked parse.\u0001, also puts the escape into the text. The check then says "maybe tagged", the hook stays, and it changes nothing.How it is verified
tests/test_typed_batch_guard.py. It counts the hook factory, theobject_hookargument ofjson.loadsand the batch elements that are read. A result of plain values (three batches forto_list(), a row per crossing foriter_dicts()) must use no hook and never read element 1. Every kind of value the bridge tags (Decimal, big integer, dates, datetimes, instants, sets, NaN and the infinities, float and double arrays, a map with an int key or the tag as key, tagged values nested in lists and maps) must keep the hook. One tagged row among 2,100 must hook exactly one batch.to_list()anditer_dicts()are also compared with the old always-hooked decoding of the same batches over each shape alone, a mixed result, the wide table oftest_typed_rows.pyand batch boundaries. The counting tests fail on the old decoder (every batch gets a hook and a read of the side array) and pass on the new one. The file is not collected on the native wheel (NATIVE_NOT_APPLICABLE).compile()of the sources, the realresults.pyagainst fake batches (the old code hooks every batch, the new one only the tagged ones), and a differential run of 30,000 random TypedRows-shaped texts, with U+0001 and the escape text inside strings, against the always-hooked decoding. Every result was equal. Those scripts are not in the PR.test-jvm.ymlskips the platform matrix for a draft, so the JVM tests have not run yet. They run when the PR is marked ready for review.Related issues
Part of #315. It stays open: nothing here claims a speed result.
Performance changes
Not measured yet. The speed claim waits for a timed A/B of a candidate wheel against 26.11.1.post2 (interleaved runs, a control run of the baseline against itself, the median ratio with its interval).
Docs
No page or
README.mddescribes how a batch is decoded, and no behavior changes, so none is updated.Checklist
uvx pre-commit run --files <changed files>passes (I ran black, isort and ruff with the versions pinned in.pre-commit-config.yaml, not pre-commit itself; the CI pre-commit job covers it)🤖 Generated with Claude Code