Repository navigation
benchmarks: the lean HTTP client is the served ArcadeDB headline, the vendor driver is not printed - #578
Merged
Conversation
… vendor driver is not printed (#577) DECISIONS #268 and #285. make_paper_tables.load_canonical keeps the rows of the benchmark's own client and sets the vendor driver's rows aside before any dedupe, table, gate or claim, so runs_paper.csv, the paper tables and the page hold lean rows only. export_web drops the client twin plumbing, names the benchmark's own client with its source and says the other engines use their vendors' clients, refuses a frozen row or an overlay file that names another client, and says on the documents transaction table that the image-defaults arm (no lean job) was not measured through the benchmark's client. page_check gets a banned-term gate for the driver's name with a REMOVE WHEN note. Fixture, tests and the four docs that named the driver are updated. Closes #577. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added a commit
to humemai/mmdbbench
that referenced
this pull request
Nov 8, 2026
…rved ArcadeDB headline (#17) Mirrors humemai/arcadedb-embedded-python#578 (DECISIONS #268 and #285): the page names the client of the served ArcadeDB rows and no longer the vendor driver. Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898
added a commit
that referenced
this pull request
Nov 8, 2026
#578 made the lean http.client client the served ArcadeDB headline and removed the official-driver twin plumbing from the exporter and the freeze. Kept from this branch: the arm identity (native and JVM builds, per-build keys, overlay build filter), the client-arcadedb image layer with the driver pinned at 0.4.0, and the PostgreSQL-wire twin jobs. Taken from main: the client wording on the page, the banned-term gate, the set-aside of other-client rows. The freeze now also sets the wire-twin rows aside (they are measured, not published), the fixture keeps main's rows with the 0.4.0 driver stamp, and the two tests that asserted the driver headline now assert the lean headline. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #577
This implements DECISIONS #268 and #285 in the generators. Every served ArcadeDB headline cell now comes from the benchmark's own lean
http.clientclient, and the vendor's driver is neither printed nor named on the page or in the paper tables.What changed:
make_paper_tables.load_canonicalsets the driver's rows aside before any dedupe (by roleheadlineor a client name containingarcadedb-driver) and keeps the lean rows.runs_paper.csv, the paper tables T2 to T5, the gates and the claims therefore hold lean rows only. Theinclude_twinsswitch and the role in the dedupe key are gone. The dense multipass overlay builder applies the same rule to a driver overlay file.export_web.pyloses the twin split,_attach_twin,client_twinand thetwin_rowsparameters._arcadedb_client_notenow nameslean_http (http.client, keep-alive)with its sourcebenchmarks/experiments/lean_http.py(also added tosource_urlsof every table that prints a served ArcadeDB row) and says the other engines are measured through their vendors' own Python clients, so client weight differs between the served engines. The exporter refuses a frozen row, a table, or a multipass overlay file that names another client.arcadedb_imgdefaults_serveron the documents transaction table, has no cell any more. The table says it was not measured through the benchmark's own HTTP client and declares the absence. I did not invent a number for it.page_check.pyhas a banned-term gate forarcadedb-driverandofficial ... driver/clientin any payload string, with aREMOVE WHENnote (upstream ships the transport fix,ArcadeData/arcadedb-drivers/next: the dense table (l3d) cannot land: page_check and the generators refuse the October route #188 merged and released).test_export_client_twin.pyis rewritten for the new behaviour,test_publish_2611_e2e.pyandfixtures/publish_2611/build_fixture.pyfollow (the fixture keeps both clients in the raw rows, one arm with driver rows only, and the committedruns.jsonl.gzis regenerated with--patch).PROTOCOL.md,CAMPAIGN.md(rows 72 and 82),COMPARATORS.mdand thelean_http.pydocstring describe the lean client as the headline without naming or comparing the driver. The pin string inCOMPARATORS.mdstays because the client image still installs the package.Which numbers move: every served ArcadeDB cell (documents, graph, sparse, dense, time series, cross-model, lifecycle, restart) is now the lean client's median. Latency columns drop and throughput columns rise on the one-row paths, memory and CPU columns of the served arms change, best-in-column marks and rank statements can change, and
claims_check.pypins and the paper's prose numbers need a re-read after the next freeze. I have not run the freeze on real rows. The default oflean_http.Session()is unchanged, so a running pass does not split its rows.Nothing of the page is published yet. Not in this PR:
runner.py, the harness, the bindings guide table inbindings/python/docs/guide/server.md, and the public methodology line in the dbbench repo.🤖 Generated with Claude Code