Make protocol benchmark accounting and timing trustworthy

[ebursztein]

Outcome

One bounded correction for protocol benchmark truth, replacing #147/#148/#149/#151. Do not use a refactor or higher throughput result as a substitute for correct measurements.

Current evidence

On main 914c72e, capsem-bench/src/protocol.rs still has separate HTTP/DNS split/spawn/join timing, starts the clock before spawning, returns one DNS bind-error sample for an assigned batch, and computes total throughput from configured requests. The protocol summarize function has no direct calls from tests found in this audit. The newer stats::Summary tests cover a different aggregation layer and do not resolve this report.

Required work

• Distinguish planned work, attempted requests, successful/failed requests, setup failures, retries and actual transferred bytes. DNS setup failure must reconcile accounting without presenting unsent work as achieved request throughput.
• Define cold-start and steady-state timing explicitly. If measuring steady state, synchronize ready workers and exclude deliberate warmup; preserve a separately named cold metric where wanted. Do not quietly change existing artifact semantics.
• Use one shared scheduling/timing owner for HTTP and DNS where behavior is truly common; keep protocol-specific setup explicit.
• Directly test protocol summarize with known samples: success, explicit transport errors, status mismatch, missing required text, byte mismatch, retries, empty/zero-duration inputs and secret-marker flags. Assert the full count/error/byte/rate result, not just serialization.
• Preserve bounded concurrency, failure propagation and host/guest comparability. Update artifact consumers/docs if schema meaning changes.

Proof and cost

Use synthetic samples and injected worker setup failures with a controlled clock/barrier; small deterministic tests should run sub-second where practical. One focused before/after protocol run is sufficient to validate interpretation. No long benchmark run added to every unit test and no repeat full gate per fix.

Out of scope: tuning HTTP/MCP/DNS performance (#96/#94/#93), porting legacy modes (#95), storage architecture and production telemetry export. Original #147/#148/#149/#151 retain detailed historical evidence; they close as superseded, not fixed.