perf(sync): parse a pull page with one jq, framed by NUL, with no eval (#908, #940)

[fujibee]

Part of #908, item (3) — and it closes #940, because the fix and the framing are the same decision. Stacked on #939 (base main so CI runs; until #939 lands the diff shows its commit too).

One jq per page, and no eval at all

storage_sync_apply_pull paid a printf | jq per message, and the jq's output was a list of shell assignments fed to eval. That second half is why #930 had to route all eighteen fields through tostring | @sh: quoting discipline was the only thing between a server-chosen string and the shell.

Now a single jq -j -R -s parses the whole page and emits, after a leading record count, each record's eighteen fields as raw values separated by NUL bytes; the loop reads them with IFS= read -r -d '' into plain variables. There is no eval — nothing server-chosen is parsed as shell anymore, so the class #930 defended against is gone rather than guarded. --raw-input keeps the old "one JSON value per line" refusal: each line is fromjson'd on its own, and a line carrying two values fails with the line named, exactly as the per-line jq refused it.

Blast radius is unchanged. Today a single bad record already refuses the whole page before anything is written (jq_ok=0 → the SQL file is discarded → 13); the page-level jq refuses the same way, and names what the old sentinel could not: which record, and which field. The stream is also verified end-to-end — the leading count must match the records read, every record must complete its eighteen frames, and one more read past the count must find EOF.

Why a NUL can never break the framing: U+0000 is refused (#940)

NUL-delimited framing is only sound if a value cannot contain a NUL — and it cannot, because jq refuses any record whose field contains U+0000, naming the record and the field. That is #940's fix as much as the framing's precondition. Measured on main before this change: a body of "xy" was accepted, reported imported, and stored as four different bytes (hex 785C3079 — the NUL re-spelled on its way through jq -r's raw output and the shell's own NUL-stripping, both of which vary by version). A value the store cannot hold verbatim is now refused, not silently rewritten. Contract cases pin both: the refusal (record and field named, nothing committed) and the framing checks (two values on one line, mid-frame truncation, bytes past the declared count).

bash cannot hold a NUL in a variable at all (command substitution and read both drop or stop at it), which is also why the alternative — carrying such a value through — does not exist: any scheme that lands in shell variables loses the byte. Refusing is the honest version of what every framing here would do anyway.

Before and after, same machine, same method

tests/perf/join-harness.sh --messages 400, bootstrap.apply, two runs each:

	ms/msg
#939 (8cb6cad)	23.6 / 23.7
this change	15.5 / 16.9

About −7 ms per message — the printf and the jq — for ~2 more minutes off a 17,300-message pull; the whole apply path has gone 280 → ~16 ms/msg across #925, #939 and this. The suite (tests/test_remote_sync.bats, 43 cases) is green under bash 5 and under /bin/bash 3.2 — the day #925 landed, a bash-3.2 divergence is what CI caught, so both are checked before pushing, and the primitives this leans on (read -r -d '', process substitution, a dynamic variable name to read) were probed on 3.2.57 directly.

Scope

• scripts/drivers/storage/sqlite-sync.sh, storage_sync_apply_pull only; the per-ack jq in reconcile is perf(sync): one jq per acknowledgement, not six forks #927's shape and untouched; the jsonl driver has its own apply.
• After this, apply's per-message process count is zero: perf(sync): quote each pulled message's fields once, not at 42 fork sites (#908) #925 took the 32 quoting forks, perf(sync): check a pulled wire id without a process (#908) #939 the uuid printf|grep, this the printf|jq. What remains per message is bash and SQLite.