benchmarks: the strict class replaces only the flush item of the native server command

benchmarks/experiments/CAMPAIGN.md

@@ -230,6 +230,7 @@ Rules every campaign keeps:
 | 90 | The ArcadeDB version a cell runs is the stage's pin, read out of what will run before the first cell, and no ArcadeDB image tag floats (humemai/arcadedb-embedded-python#470). `runner.py` takes the pin from the `ARCADEDB_WHEEL` file name and, for every ArcadeDB arm of the batch, reads the version out of the image that will run (embedded: the installed `arcadedb-embedded` version and the engine jar the wheel ships; served: the engine jar of the server image) and refuses to start on any difference, or when the batch has an ArcadeDB arm and no pin; the readback is in the manifest (`engine_pin`). The embedded image is `dbbench:arcadedb-<version>`, written by `build_images.sh`, and the runner refuses every other ArcadeDB tag (`dbbench:arcadedb` stays only as the logical build target name in `runner.BACKENDS`); the comparator images keep `dbbench:<target>`, their engines being pinned by `==` or digest and read back by `version_pin_check.py --runtime` | the embedded and served ArcadeDB arms of every lane | built, verified by unit tests | `engine_pin.py`, `test_engine_pin.py`; the trigger: the laptop's `dbbench:arcadedb` still held 26.10.1 beside `dbbench:arcadedb-26102` at 26.10.2 (2026-10-22, `.notes/bench/repros/ts-doc-oom-20261022/`), and every earlier check read rows after the run |
 | 91 | The composed embedded stack joins the cross-model lane: SQLite (documents) + LanceDB (vectors, fp32 `IVF_HNSW_FLAT`) + LadybugDB (graph) in one Python process, the default trio of the Cognee-style agent-memory builders and the real alternative to ArcadeDB for that audience (the owner, 2026-10-25) | `composed_embedded_sqlite_lance_ladybug`, the `e2` lane (hybrid in both durability classes, and the atomicity workload) | **built and dry-run on the laptop (2026-10-25), untimed**: `ladybug==0.21.2` added to the dense image's package line (an existing pin, no new version; 0.22.0 since 2026-10-25, row 92), the arm in `runner.BACKENDS` and `LANES["e2"]`, an `INDEX_DECISIONS` entry, the generator's `composed-embedded` engine (Tier 1, after `qdrant-neo4j`) and the regenerated `stage_drafts_2611.sha256`. Both ends of the lane's checks pass on its rows: the retrieval, hop and filtered-candidate digests equal LadybugDB's and DuckDB's at e2, 40 of 40 atomicity trials torn (nothing spans the stores), F10 and F14 clean | `COMPARATORS.md` "The composed embedded stack"; `repros/composed-embedded-lance/probe.py` |
 | 92 | LadybugDB moves to 0.22.0 on the client and dense images, and its `vector` extension is baked into the client image at build time, pinned by sha256 | every LadybugDB arm: `ladybug_graph`, `ladybug_dense`, `ladybug_e2`, `composed_embedded_sqlite_lance_ladybug`, `ladybug_lifecycle` | **built (2026-10-25)**: `build_images.sh` (`ladybug==0.22.0`, `LADYBUG_EXTENSIONS`), `Dockerfile.bench`, `bake_ladybug_extensions.py`, `test_ladybug_pin.py`. 0.21.2 can no longer `LOAD vector` (the vendor overwrote the v0.21.0 download at 04:28 UTC on 2026-10-25, LadybugDB/ladybug#1390). Untimed laptop cells on 0.22.0: dense micro with the mutation pass (0.20.4 control dies at the re-insert, 0.22.0 completes), graph micro, e2 (`ladybug_e2` and the composed arm). Open: the fdatasync counts in `bench_common` were measured at 0.21.2 and are not re-measured on 0.22.0; the DuckDB extensions are not baked yet (COMPARATORS.md, "Runtime extensions") | PR on humemai/arcadedb-embedded-python; `.notes` repros/ladybug-vector-ext-20261025 |
+| 93 | The strict class passes the native server command line through unchanged except for the flush item: `durability_server_patch` replaces only the argument that is `-Darcadedb.txWalFlush=0` by the strict value, because every item of a native `server_cmd` is one argument and the helper that appends the token to a string had glued it onto `-Xmx` and every other `-D` item, so the native image refused to start (humemai/arcadedb-embedded-python#573) | the strict class of every served ArcadeDB native arm (the JVM served arms and the relaxed class are not affected) | **built, not in the running pass:** the four-arm pass holds its pin and does not sync; the fix is part of the commit of the planned catch-up, which measures the strict native server cells of the passes already taken (6 cells per pass) as new rows at that pin | `runner.py`, `test_native_strict_argv.py` (fails on the old code), #573 |
 
 **What the 2026-11 instrument measures (DECISIONS #196).** The harness measures one instrument and every method choice named here is the behaviour, not a switch. (1) The sparse lane warms up every engine on queries apart from the timed set, and the runner holds the CPU idle-exit latency at 10 microseconds for each timed cell (row 81); every timed cell of the pass also starts behind a pre-cell temperature gate (PROTOCOL.md section 2). (2) The dense lane indexes the vector id for the delete on every engine that would scan (row 84). (3) The served ArcadeDB headline is the official `arcadedb-driver`, with the lean HTTP client as the twin sensitivity row (rows 72, 82). (4) The l4 queries with large answers read each engine on its documented bulk path: for `q_high` and `q_groupby` ArcadeDB embedded `to_arrow()`, DuckDB `to_arrow_table()`, SQLite `fetchall()` (no columnar path), with `to_list()` / `fetchall()` timed beside it as `q_high_plain_ms` and `q_groupby_plain_ms`; rows stamp `read_path`, `read_path_queries`, `read_path_plain`, and `read_path_sensitivity`; small results do not move; a missing pyarrow or bridge jar falls back to `to_list()` and the row says `read_path_arrow_missing`; the typed read's digest must equal the plain read's after the lane's declared coerce, and a mismatch ends the cell. (5) The ArcadeDB arms, embedded and served, leave `-Xms` unset (the bindings' documented practice and the maintainers' guidance for a fixed memory budget; DECISIONS #200 for the served arms) and keep `-Xmx` from the cell's memory budget; Elasticsearch and Neo4j keep the initial heap they have (PROTOCOL.md section 7, "JVM initial heap"). `BENCH_ARCADEDB_TS_NDJSON` stays off (row 85, measured mixed). Every row stamps `instrument` = `2026-11`; a `BENCH_INSTRUMENT` of anything else is refused, the runner forwards its own value to every cell so a container cannot disagree with the launcher, and `test_instrument_method.py` holds the list. The 2026-10 and September instruments are retired: no cell is measured with them again.
 

benchmarks/experiments/runner.py

@@ -2242,7 +2242,11 @@ def durability_server_patch(cfg, cls):
             notes.append(f"txWalFlush={bench_common.ARCADE_STRICT_TX_WAL_FLUSH}")
     # ArcadeDB served from the native image: the same token, on the command line.
     if any(isinstance(c, str) and c.startswith("-Darcadedb.txWalFlush=") for c in cmd):
-        cmd = [_set_tx_wal_flush(c, bench_common.ARCADE_STRICT_TX_WAL_FLUSH) if isinstance(c, str) else c for c in cmd]
+        # Replace only the item that IS the token. _set_tx_wal_flush appends the token to a string that lacks one, and every item of a
+        # native command is one argv entry, so mapping it over all items glued " -Darcadedb.txWalFlush=1" onto -Xmx and every other
+        # -D item (the native runtime then read "4g -Darcadedb.txWalFlush=1" as the heap size and the server never started).
+        cmd = [f"-Darcadedb.txWalFlush={bench_common.ARCADE_STRICT_TX_WAL_FLUSH}"
+               if isinstance(c, str) and c.startswith("-Darcadedb.txWalFlush=") else c for c in cmd]
         notes.append(f"txWalFlush={bench_common.ARCADE_STRICT_TX_WAL_FLUSH}")
     # QuestDB: its commit mode is a server setting.
     if "questdb" in str(cfg.get("server_image", "")):

benchmarks/experiments/test_native_strict_argv.py

@@ -0,0 +1,63 @@
+"""The strict class on the native server arm: the argument list the container start receives, with no container (pure Python).
+
+The native server image is started with its settings as command-line arguments, and each item of `server_cmd` is ONE argument. The strict
+class's durability patch used to map a helper that appends the flush token to any string lacking one over every item, so the argument
+list became ["-Xmx4g -Darcadedb.txWalFlush=1", "-Darcadedb.server.mode=development -Darcadedb.txWalFlush=1", ...] and the native
+runtime refused the heap size ("Invalid option '-Xmx4g -Darcadedb.txWalFlush=1' does not specify a valid number", server_not_ready).
+test_native_server_arm.py only counted the items that START with the token, which the damage never touched.
+Run with `python -I -m pytest test_native_strict_argv.py -q` from this directory.
+"""
+import os
+import sys
+from pathlib import Path
+
+import pytest
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+os.environ.setdefault("BENCH_INSTRUMENT", "2026-11")
+
+import runner as R  # noqa: E402
+
+IMAGE = "arcadedata/arcadedb:26.11.1-native"
+FLUSH = "-Darcadedb.txWalFlush="
+# Copied at import: other test modules rebuild R.BACKENDS (the gated pq arms come and go), so a name looked up later may be gone.
+CFG = {n: dict(c) for n, c in R.BACKENDS.items()}
+SERVED = sorted(n for n, c in CFG.items()
+                if "arcadedb" in n and c.get("topology") == "client_server"
+                and str(c.get("server_image", "")).startswith("arcadedata/arcadedb") and not c.get("jvm_defaults"))
+
+
+def native_argv(name, cls, heap="4g"):
+    """The argument list the runner passes after the image name for a native cell of this arm and class (the runner formats `{heap}`)."""
+    cfg, _ = R.durability_server_patch(R.native_server_arm(CFG[name], IMAGE), cls)
+    return [c.format(heap=heap) for c in cfg["server_cmd"]]
+
+
+def test_there_are_served_arms_to_check():
+    assert {"arcadedb_server", "arcadedb_graph_server", "arcadedb_e2_server"} <= set(SERVED)
+
+
+@pytest.mark.parametrize("name", SERVED)
+@pytest.mark.parametrize("cls", ["relaxed", "strict"])
+def test_every_item_of_a_native_command_is_one_argument(name, cls):
+    argv = native_argv(name, cls)
+    assert argv[0] == "-Xmx4g"
+    assert all(isinstance(c, str) and c and not any(ch.isspace() for ch in c) for c in argv), argv
+    assert all(c.startswith("-D") for c in argv[1:]), argv
+
+
+@pytest.mark.parametrize("name", SERVED)
+def test_strict_changes_only_the_flush_item_of_the_native_command(name):
+    relaxed, strict = native_argv(name, "relaxed"), native_argv(name, "strict")
+    assert [c for c in strict if c.startswith(FLUSH)] == [FLUSH + "1"]
+    assert len(strict) == len(relaxed)
+    assert [(a, b) for a, b in zip(relaxed, strict) if a != b] == [(FLUSH + "0", FLUSH + "1")]
+
+
+@pytest.mark.parametrize("name", SERVED)
+def test_the_jvm_arm_still_takes_the_strict_flag_as_one_token_in_java_opts(name):
+    base = CFG[name]
+    cfg, note = R.durability_server_patch(base, "strict")
+    opts = [e for e in cfg["server_env"] if isinstance(e, str) and e.startswith("JAVA_OPTS=")]
+    assert len(opts) == 1 and opts[0].count(FLUSH) == 1 and FLUSH + "1" in opts[0].split() and "txWalFlush=1" in note
+    assert cfg["server_cmd"] == list(base.get("server_cmd", []))