Repository navigation
bench(gate): widen the url-parse-ipv4 cap to 20.0 for bimodal CI runners - #498
Merged
Merged
Conversation
Across 28 ubuntu-latest CI bench-gate runs (median of 3 launches per side, Node v26.13.0) url-parse-ipv4 measured 10.37x-16.06x and breached the 14.7x cap 3 times with no runtime change: Node's time on that bench is either ~12 ms or ~5-7 ms depending on the runner, Lava's moves less. Cap set at ~1.4x the max observed; measurements and the detection cost are recorded in thresholds.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…' fire-proof stale perf-memory-auditor re-read all 28 CI rows: 20.0 (~1.25x the max, >5 sd above the fast-runner mean) has no false positive among them and keeps more detection than 22.5. The note no longer claims no runtime change between breaches (5228c42 moved the parser) and gives the evidence that the breaches track the runner instead. bench/micro/url.js no longer claims the caps are proven to fire: the old hand slowdown breaches none of the current caps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Widens the
url-parse-ipv4bench-gate cap from 14.7x to 20.0x, at the owner's request.bench/thresholds.jsonis on the integrity-protected list, so this change was made with the owner's explicit approval. The PR also corrects the stale "caps proven to fire" claim inbench/micro/url.js. No runtime code changes.Why
The cap breached on master after #496 (16.06x) and on #497 (15.01x), even though both PRs touched only
scripts/and the mutation manifest.Across all 28 ubuntu-latest CI runs with a bench-gate row (runs 43938992938 through 44067776896, median of 3 launches per side, Node v26.13.0), the ratio ranged from 10.37x to 16.06x and breached 14.7x three times. Those three breaches are the runs with the lowest Lava time of all 28 (77-83 ms). They breach only because Node was faster still on those runners.
Node's time sits in two clusters:
So the ratio follows the runner, not Lava. The full figures, with commit references, are in the
"//url"note.The value
The value is 20.0, not the #412 convention of ~1.4x the max (22.5). That convention was set from only 3 samples. With 28 samples, 20.0 is ~1.25x the max and more than 5 sd above the fast-cluster mean, it would have produced 0 false positives across the 28, and it keeps more detection.
Confidence that 20.0 stays green is medium. A false trip needs Node at ~4 ms, which is about 19% faster than the fastest of the 28 runs.
Also
bench/micro/url.js:33claimed the url caps were "PROVEN TO FIRE". The hand slowdown cited there (ipv4 15.35x) breaches none of the current caps. The comment now says that proof is stale.Follow-up, not done here: the bench runs 50k iterations, so Node finishes in ~5 ms and launch noise dominates. More iterations might allow a tighter cap. This is untested.
Gates (
/pr-gate, verdict SHIP: P0 0, P1 0)Reviewers:
bench/run.mjs, and nothing pins 14.7.Fix-round edits (d3fce98) were made by the conductor directly; they are data and comment text only.
🤖 Generated with Claude Code