perf(encode): 1.08-1.27x faster matchfind at L3-L12, byte-identical; rusty_alloc 2.0.5 - #12
Merged
Merged
Conversation
The chain finders' three hot units were priced against the emitted assembly and worked round-robin: the walk's per-CANDIDATE first-word path, the fill's per-INSERTED-BYTE loop, and the lazy finder's per-POSITION loop. Every verdict is a shortest header-to-latch path on the emitted asm, never a stopwatch -- this box runs at ~65% load from other work and cannot resolve one. The wall-clock check, taken with ONE measurement program compiled against both the released v0.2.3 and this tree, whole processes alternated ABBA and pinned to one core, estimator the floor. Instrument floor measured first by running a binary against itself: encode +-3%, decode +-0.5%. L5 Greedy 67.2 -> 79.1 MB/s (dickens) 101.8 -> 128.7 (samba) L7 Lazy 11.0 -> 13.4 22.1 -> 28.0 L9 Lazy2 18.2 -> 20.9 38.0 -> 46.0 L12 Lazy2 3.5 -> 3.9 12.3 -> 13.3 L1 Fast flat -- the ladder that took no bricks L15/L19 flat to 1.07 -- one brick, a per-call prologue trim The win is exactly where the campaign worked and absent where it did not, which is the reason to trust both the counts and the clock. Compressed output is byte-identical throughout: GOLD 2F6594F7EEDBD12B / 59,680,638 bytes and LDM 57BE83EA4E1199E8 / 57,796,847, unchanged after every landed brick. Sizes move at L3 only, where the DFast back-extension trades: samba -1.14% (smaller), dickens +0.04%. Also in this commit: - the allocator seam pin moves 1.1.4 -> 2.0.5. Priced on the same instrument and it is NEUTRAL on this workload (bulk 17/40 wins, z = -0.95; an allocation-heavy arm run separately because bulk codec loops are the wrong place to judge an allocator reads flat too). Recorded as a currency update rather than dressed up as a win. - a kernel-reach gate in CI that COUNTS whether the SIMD kernels are actually called, with a poison self-check that must fail. Every slot reads 100.00%. The census exists because an AVX2 checksum kernel once shipped reachable only from a test and a benchmark, and every other gate passed. - the library modules the tree was already using untracked (copies, kreach) and their gates, so a clean checkout builds. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deterministic censuses and the assembly-census tools, kept because a number in the changelog that nobody can reproduce is an assertion, not evidence. None of this is in the published crate: `rusty_zstd-bench` is publish = false and CI checks it with `--all-targets --features profile`. The ones the changelog names by hand: - `speedab` -- the A/B measurement program: loops compress/decompress in memory with no file I/O in the timed region, best-of-N per run, so the SAME source compiled against two library versions is the whole comparison. - `fillcensus` -- walk exits, candidates examined, tag skips (and false skips, which must read 0), fill inserts, and the phantom position-0 census. - `phantoms` -- candidates examined at position 0 and how many were accepted, which is what settled the null-link representation question. - `tools/asmcensus/` -- the board (`verdict3.py`), the one-number per-byte model (`score.py`), the path dumps, and `fillloops.py`, which reads a fill body's arms apart when the board's single row folds them together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ible Both files were untracked before this branch, so no CI had ever compiled them. Committing them is what exposed the defects -- which is the argument for tracking instruments rather than leaving them on one machine's disk. - `rssgrow` defined `fn rss()` under `#[cfg(windows)]` and called it unconditionally, so every Linux and macOS build failed with E0425. The body is a stub on every platform, so the gate protected nothing. Removed. - the kernel-reach gate asserted `checked >= 6` exercised sites. Eight of the ten slots need BMI2, which no aarch64 host has, so on macOS runners they are SKIPPED by design and a perfectly-routed build failed the gate. The floor is now derived from the host (6 with BMI2, 2 with only a vector ISA, else 1) so it still refuses to pass on silence without asserting an x86 assumption. The poison self-check still bites on aarch64: `set_xxh_avx2_arm(false)` gates the NEON stripe path as well as the AVX2 one, so `xxh64 stripes` goes scalar and the routing assertion fires. `count_eq_len`'s NEON arm is chosen at compile time and cannot be poisoned; one poisonable slot is enough, and that asymmetry is now written down beside the floor. Verified by cross-checking rather than another CI round trip: `cargo check --all-targets --features profile` clean for both x86_64-unknown-linux-gnu and aarch64-apple-darwin, and locally the gate passes while the poison run fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The entry described the gate as failing "if fewer than 6 sites were exercised", which is what it did until this branch and what broke every aarch64 runner. A stale comment beside the thing it documents is the drift this repository already calls out by name, so the description moves with the code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed it The measured section credited the L3 delta to the DFast back-extension, which shipped in v0.2.3 and therefore cannot explain a difference against v0.2.3. The cause is the next-long offset-trade dispatch defaulting ON, which this release already documents at -0.36% across an 18-corpus L3 board. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The matchfind campaign, measured, plus the allocator seam moved to
rusty_alloc 2.0.5. Compressed output is byte-identical to
v0.2.3at everylevel except 3.
The numbers
Every verdict in the changelog is an instruction count, because this box
runs at ~65% load from other work and cannot resolve a stopwatch. This is the
wall-clock check on those counts: one measurement program (
speedab) compiledagainst both the released
v0.2.3(e672cd0) and this branch, loopingcompressin memory with no file I/O in the timed region, whole processesalternated ABBA and pinned to one core at High priority, estimator the floor.
The instrument's floor, measured first by running a binary against itself:
encode ±3%, decode ±0.5%. Nothing below that is reported as a result.
The shape is the reason to trust it: the win is exactly where the campaign
worked and absent where it did not. L5–L12 are the greedy and lazy chain
finders, which took nearly every brick. L1 is the Fast ladder, which took none.
L15–L19 are the tree ladder, which took a single per-call prologue trim.
The allocator, priced honestly
rzstd-allocmoves its exact pin1.1.4→2.0.5. Measured on the sameinstrument with the same codec on both arms:
Every cell is inside the floor and the sign test is a coin flip, so this is
recorded as a dependency-currency update, not a speed win. The
allocation-heavy arm was run separately because bulk codec loops are the wrong
place to judge an allocator; it reads flat too.
Correctness
2F6594F7EEDBD12B/ 59,680,638 and LDM57BE83EA4E1199E8/ 57,796,847 — unchanged after every landed brick andagain after the allocator bump.
in this release (the next-long offset-trade dispatch defaulting ON): samba
−1.14% (smaller), dickens +0.04%. That is also why the L3 decode row
moves — a smaller frame is less work, not a decode win, and is not claimed.
clippy -D warningsandfmt --checkclean on the three CIcrates; wasm32
stdandallocchecks clean;cargo package -p rusty_zstdbuilds from committed state.
called, with a poison self-check that must fail. Every slot reads 100.00%.
It exists because an AVX2 checksum kernel once shipped reachable only from a
test and a benchmark, and every other gate passed.
CI found two defects, and that is the point
Both files below were untracked before this branch, so no CI had ever
compiled them. Committing the instruments is what exposed them:
rssgrowdefinedfn rss()under#[cfg(windows)]and called itunconditionally — every Linux and macOS build failed with E0425.
checked >= 6. Eight of its ten slotsneed BMI2, which no aarch64 host has, so macOS runners skip them by design
and a correctly-routed build failed. The floor is now host-derived.
Fixed and cross-checked locally against
x86_64-unknown-linux-gnuandaarch64-apple-darwinrather than by another CI round trip.Note for review
The second commit adds instruments and
tools/asmcensus/, and the first addstwo library modules (
copies,kreach) plus two gates that the tree wasalready using untracked — without them a clean checkout does not build.
Merging this to
mainis what triggers release-plz to open its release PR.🤖 Generated with Claude Code