Skip to content

Native find_nearest: send the query vector as one float32 array, not a Python list - #203

Merged
tae898 merged 4 commits into
mainfrom
perf/native-find-nearest-ndarray
Nov 4, 2026
Merged

tae898 merged 4 commits into
mainfrom
perf/native-find-nearest-ndarray

Conversation

@tae898

@tae898 tae898 commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

NativeVectorIndex._run_search (native find_nearest and find_nearest_by_key) turns the query into a Python list with tolist(). Since #107 the codec then either checks that list for float32-exactness (_exact_float_list: set(map(type, ...)), np.array, astype, array_equal, then the side-buffer tag BL), or, for a float64 query whose values float32 rounds, prints one float repr per element as JSON text (about 611 us at 1536 dimensions, the figure in the codec's own docstring). This change hands the codec a float32 ndarray instead: a numeric 1-d query (an ndarray of any numeric dtype, or a list or tuple of numbers, which numpy walks in C) goes through the float32 side buffer (tag B) directly. The engine reads the vector as float32 whatever it is given, so the rounding does not change. Text or None entries, a generator, ragged input and a missing numpy stay a list and reach the engine as before.

This is the port of humemai/arcadedb-embedded-python#523 (backlog row N-07) onto main. It was written before #107; what #107 already removed is the per-element _prep calls that #523 counted (768 to 0). What is left to save is tolist(), the exactness pass for a float32 query, and the JSON text for a float64 query. Whether that shows in a search of 0.3 to 0.75 ms is for the timed A/B to say; no timing was run here.

Results are identical: every form of the query (float32 and float64 array, tuple, list of numpy floats, generator, non-contiguous view, integer array, find_nearest_by_key, a float64 query that float32 rounds) answers the same neighbours at the same distances (exact equality), allowed_rids and ef_search still apply, and a query of the wrong dimension or with None or text entries is still refused with ArcadeDBError.

Tests: tests/test_find_nearest_query_forms.py runs on both backends and passes before and after the change (it is the contract the change keeps). tests/test_native_find_nearest_params.py is native only (not collected on the JVM backend): where the floats travel (tag B with dim floats beside the text, never BL) and that _exact_float_list is not run on a query; 5 of its 6 tests fail on the code before. Local check on the committed bytes: the 14 tests pass against the unmodified native wheel 26.11.1.dev0 (matched dev pair dd45b53 + e353330c5c) with only this PR's schema_objects.py copied over it (not a wheel built from this repo); the vector, SQL-vector, numpy and query-vector-parameter suites show the same failing set with and without the change (the wheel predates main, and test_vector.py needs JPype). black, isort (profile black), pyflakes and bandit are clean on the changed files. Not run: the full suite, scripts/build.sh, CI (a draft skips the matrix).

This stays a draft until a timed A/B decides it: native wheels of main and of this head, find_nearest k = 10 p50 at 768 and 1536 dimensions for a float32 array, a float64 array and a list, with the answers compared pair by pair. If the interval does not exclude 1.00 it stays a cleanup or is closed. It touches the same function as the lazy-vector change of the same series (a separate PR); the two edit adjacent lines of _run_search, so whichever merges second needs a trivial rebase.

Refs #19

🤖 Generated with Claude Code

tae898 and others added 2 commits November 3, 2026 08:23
…a Python list

NativeVectorIndex._run_search turned every query into a list with tolist(). The codec then checked the list for float32-exactness
(_exact_float_list: a pass of numpy over the list, tag BL) or, for a float64 query whose values float32 rounds, printed JSON number
text (about 611 us at 1536 dimensions, #107). A numeric 1-d query (an ndarray of any numeric dtype, or a
list/tuple of numbers, which numpy walks in C) now goes to the codec as a float32 array and travels through the float32 side buffer
(tag B) directly. The engine reads the vector as float32 whatever it is given, so the rounding is unchanged. Anything else (text or
None entries, a generator, ragged input, numpy missing) stays a list and reaches the engine as before.

Port of humemai/arcadedb-embedded-python#523 (backlog row N-07) onto main after #107: the per-element _prep calls it removed are
gone since #107, what remains is tolist() plus the exactness check, and the float64 text path. Untimed; a timed A/B decides it.

Tests: test_find_nearest_query_forms.py (both backends: every form answers the same neighbours at the same distances, refusals
unchanged; passes before and after) and test_native_find_nearest_params.py (native: where the floats travel and that the list
exactness check is not run; 5 of its 6 tests fail on the code before).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No textual conflicts: main's #202 (record ids as a :rid parameter, src/arcadedb/_native/records.py and database.py) and #201 (run_in_transaction witness) touch other hunks than this branch. The record-id rule of #202 is kept: this branch pastes no record id into SQL text (the lazy per-hit vector load, if present, goes through lookup_by_rid, which keeps its literal id by design, see tests/test_native_statement_texts.py).

Static checks after the merge: compile() and pyflakes of the changed files (the unused typing.Optional in records.py is on main too), pre-commit black and isort pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898

tae898 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

Timed A/B (native wheel; arm A = main's CI native wheel, arm B = the same tree with only this PR's files, A/A control; laptop timed lane, tools/ab.py --noise, ABBA, round 0 dropped). find_nearest p50 latency, B/A median with 95% CI (below 1 = this PR is faster):

query passed as 1536-d 256-d
float32 array 0.951 [0.945, 0.954] 0.952 [0.942, 0.96]
float64 array 0.616 [0.614, 0.621] 0.777 [0.774, 0.781]
list of float32 values 0.975 [0.964, 0.982] 0.969 [0.966, 0.977]
list of doubles 0.631 [0.627, 0.635] 0.792 [0.787, 0.798]
statement path (control, not changed by this PR) 1.003 [0.995, 1.014] 1.004 [0.991, 1.006]

Clean pairs per figure: 11, 12. Every counted run returned the same ids and distances on both arms (answer digests equal in every round), the encoder spy showed the side buffer where this PR takes it, and the statement path stayed flat. By the rule written before the run this is a SHIP: every headline form is faster on both dimensions with no regression and an unchanged control. The large gain is for float64 input (array or list of doubles): main sends it as JSON text, this PR sends it as one float32 buffer like the other forms (encoder spy on every run).

The merge waits for the end of the current batch of timed studies that pin main (merging now would move their pins); the PR leaves draft at that point.

tae898 added a commit that referenced this pull request Nov 4, 2026
…dexed vector, loaded by RID on first read

find_nearest keeps its default: every hit carries its vector, and the default search is the code of the query-as-float32 change (#203), untouched. A new keyword-only include_vector=False asks for hits built without the vector, for callers that need only ids or distances. On the native wheel the search then runs SELECT *, !<vector property>, !record FROM (SELECT expand(vectorNeighbors(...))) and each hit loads its record once by RID the first time the vector or the whole record is needed. The JVM build accepts the keyword and ignores it (it loads whole records either way).

Reworks the earlier lazy-by-default version after its timed A/B: hits not read 0.66 and 0.805 (1536 and 256 dims), but vector read 1.011 and 1.336, so lazy-by-default is a cost for callers that read the vector. Tests run on both backends (values equal lookup_by_rid, keyword accepted) and on the native wheel (default ships the vector with no lookup and no dropped column, opt-in ships none and looks up once per record).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 marked this pull request as ready for review November 4, 2026 21:42
…2 ulps, native exactly

The JVM leg of the PR run failed test_every_form_of_the_query_returns_the_same_neighbours_at_the_same_distances on one distance that differed in the last float32 bit between two calls with the same query values (0.9530812501907349 against 0.9530811309814453, same neighbour order); the same test passed on this head in the dispatched JVM run. Exact distances are not a contract on the JVM build. On the native build, the path this PR changes, the comparison stays exact, because a tolerance of float32 size could not tell a query that float32 rounds from its float32 form, which is what the float64 test guards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 54b9d02 into main Nov 4, 2026
38 checks passed
@tae898
tae898 deleted the perf/native-find-nearest-ndarray branch November 4, 2026 23:46
@tae898 tae898 added this to the 26.11.1 milestone Nov 5, 2026
@tae898 tae898 added native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement vector Vector indexes and find_nearest labels Nov 5, 2026
tae898 added a commit that referenced this pull request Nov 5, 2026
#250)

Describe the #203 behavior where users read about find_nearest: a paragraph and the A/B table in vectors.adoc (figures copied by script from the study's score file, p50 B/A with 95% intervals), a pointer from the statement-only "A query vector" section, and the query_vector Args text of find_nearest. The find_nearest_approximate Args stay as they are: its native path (_run_approximate) does not use the float32 buffer. The which-package vector-search medians predate #203, so they are labelled as measured before it instead of being replaced by numbers that were not re-measured. The sentence with the find_nearest signature is untouched so the include_vector change (#204) rebases cleanly.

Closes #249

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
tae898 added a commit that referenced this pull request Nov 5, 2026
…dexed vector, loaded by RID on first read

find_nearest keeps its default: every hit carries its vector, and the default search is the code of the query-as-float32 change (#203), untouched. A new keyword-only include_vector=False asks for hits built without the vector, for callers that need only ids or distances. On the native wheel the search then runs SELECT *, !<vector property>, !record FROM (SELECT expand(vectorNeighbors(...))) and each hit loads its record once by RID the first time the vector or the whole record is needed. The JVM build accepts the keyword and ignores it (it loads whole records either way).

Reworks the earlier lazy-by-default version after its timed A/B: hits not read 0.66 and 0.805 (1536 and 256 dims), but vector read 1.011 and 1.336, so lazy-by-default is a cost for callers that read the vector. Tests run on both backends (values equal lookup_by_rid, keyword accepted) and on the native wheel (default ships the vector with no lookup and no dropped column, opt-in ships none and looks up once per record).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tae898 added a commit that referenced this pull request Nov 5, 2026
…dexed vector (explicit opt-in) (#204)

* Native find_nearest: include_vector=False returns hits without the indexed vector, loaded by RID on first read

find_nearest keeps its default: every hit carries its vector, and the default search is the code of the query-as-float32 change (#203), untouched. A new keyword-only include_vector=False asks for hits built without the vector, for callers that need only ids or distances. On the native wheel the search then runs SELECT *, !<vector property>, !record FROM (SELECT expand(vectorNeighbors(...))) and each hit loads its record once by RID the first time the vector or the whole record is needed. The JVM build accepts the keyword and ignores it (it loads whole records either way).

Reworks the earlier lazy-by-default version after its timed A/B: hits not read 0.66 and 0.805 (1536 and 256 dims), but vector read 1.011 and 1.336, so lazy-by-default is a cost for callers that read the vector. Tests run on both backends (values equal lookup_by_rid, keyword accepted) and on the native wheel (default ships the vector with no lookup and no dropped column, opt-in ships none and looks up once per record).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* find_nearest include_vector: document the measured opt-in figures; compare hit distances like the query-form tests

The "Hits without the vector" section of vectors.adoc now carries the timed A/B of the opt-in (native wheel 26.11.1.dev0, engine 26.10.2, k = 10, 12 clean pairs per figure): p50 B/A with 95% intervals for include_vector=False (hits not read, ids read, vector of every hit read) and for the unchanged default call, at 1536 and 256 dimensions. The figures are generated from the study's score file by a script, not typed. The text says the keyword is an opt-in for callers that do not read the vector (256 dimensions: reading every vector afterwards costs more than the default).

test_the_keyword_changes_no_answer compared distances with ==, which the JVM build does not guarantee between two calls (main's _assert_same rule): ids compare exactly, distances exactly on the native build and to a few float32 ulps on the JVM build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

native The arcadedb package (native library build) performance Speed or memory of the bindings, with a measurement vector Vector indexes and find_nearest

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant