Skip to content

benchmarks: the lean HTTP client is the served ArcadeDB headline, the vendor driver is not printed - #578

Merged
tae898 merged 1 commit into
mainfrom
bench/lean-client-headline
Nov 8, 2026
Merged

tae898 merged 1 commit into
mainfrom
bench/lean-client-headline

Conversation

@tae898

@tae898 tae898 commented Nov 8, 2026

Copy link
Copy Markdown

Closes #577

This implements DECISIONS #268 and #285 in the generators. Every served ArcadeDB headline cell now comes from the benchmark's own lean http.client client, and the vendor's driver is neither printed nor named on the page or in the paper tables.

What changed:

  • make_paper_tables.load_canonical sets the driver's rows aside before any dedupe (by role headline or a client name containing arcadedb-driver) and keeps the lean rows. runs_paper.csv, the paper tables T2 to T5, the gates and the claims therefore hold lean rows only. The include_twins switch and the role in the dedupe key are gone. The dense multipass overlay builder applies the same rule to a driver overlay file.
  • export_web.py loses the twin split, _attach_twin, client_twin and the twin_rows parameters. _arcadedb_client_note now names lean_http (http.client, keep-alive) with its source benchmarks/experiments/lean_http.py (also added to source_urls of every table that prints a served ArcadeDB row) and says the other engines are measured through their vendors' own Python clients, so client weight differs between the served engines. The exporter refuses a frozen row, a table, or a multipass overlay file that names another client.
  • The arm without a lean job, arcadedb_imgdefaults_server on the documents transaction table, has no cell any more. The table says it was not measured through the benchmark's own HTTP client and declares the absence. I did not invent a number for it.
  • page_check.py has a banned-term gate for arcadedb-driver and official ... driver/client in any payload string, with a REMOVE WHEN note (upstream ships the transport fix, ArcadeData/arcadedb-drivers /next: the dense table (l3d) cannot land: page_check and the generators refuse the October route #188 merged and released).
  • test_export_client_twin.py is rewritten for the new behaviour, test_publish_2611_e2e.py and fixtures/publish_2611/build_fixture.py follow (the fixture keeps both clients in the raw rows, one arm with driver rows only, and the committed runs.jsonl.gz is regenerated with --patch).
  • PROTOCOL.md, CAMPAIGN.md (rows 72 and 82), COMPARATORS.md and the lean_http.py docstring describe the lean client as the headline without naming or comparing the driver. The pin string in COMPARATORS.md stays because the client image still installs the package.

Which numbers move: every served ArcadeDB cell (documents, graph, sparse, dense, time series, cross-model, lifecycle, restart) is now the lean client's median. Latency columns drop and throughput columns rise on the one-row paths, memory and CPU columns of the served arms change, best-in-column marks and rank statements can change, and claims_check.py pins and the paper's prose numbers need a re-read after the next freeze. I have not run the freeze on real rows. The default of lean_http.Session() is unchanged, so a running pass does not split its rows.

Nothing of the page is published yet. Not in this PR: runner.py, the harness, the bindings guide table in bindings/python/docs/guide/server.md, and the public methodology line in the dbbench repo.

🤖 Generated with Claude Code

… vendor driver is not printed (#577)

DECISIONS #268 and #285. make_paper_tables.load_canonical keeps the rows of the benchmark's own client and sets the vendor
driver's rows aside before any dedupe, table, gate or claim, so runs_paper.csv, the paper tables and the page hold lean rows
only. export_web drops the client twin plumbing, names the benchmark's own client with its source and says the other engines
use their vendors' clients, refuses a frozen row or an overlay file that names another client, and says on the documents
transaction table that the image-defaults arm (no lean job) was not measured through the benchmark's client. page_check gets
a banned-term gate for the driver's name with a REMOVE WHEN note. Fixture, tests and the four docs that named the driver are
updated. Closes #577.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898
tae898 merged commit 31fcc9b into main Nov 8, 2026
4 checks passed
@tae898
tae898 deleted the bench/lean-client-headline branch November 8, 2026 23:43
tae898 added a commit to humemai/mmdbbench that referenced this pull request Nov 8, 2026
…rved ArcadeDB headline (#17)

Mirrors humemai/arcadedb-embedded-python#578 (DECISIONS #268 and #285): the page names the client of the served ArcadeDB rows and no longer the vendor driver.

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898 added a commit that referenced this pull request Nov 8, 2026
#578 made the lean http.client client the served ArcadeDB headline and removed the
official-driver twin plumbing from the exporter and the freeze. Kept from this branch:
the arm identity (native and JVM builds, per-build keys, overlay build filter), the
client-arcadedb image layer with the driver pinned at 0.4.0, and the PostgreSQL-wire
twin jobs. Taken from main: the client wording on the page, the banned-term gate, the
set-aside of other-client rows. The freeze now also sets the wire-twin rows aside (they
are measured, not published), the fixture keeps main's rows with the 0.4.0 driver stamp,
and the two tests that asserted the driver headline now assert the lean headline.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The benchmark generators still print the official driver as the served ArcadeDB headline

1 participant