test(ci): #28 Phase 6.7 — conformance + cross-validation + ABI snapshot + reverse-dep matrix

[polaz]

Summary

Phase 6.7 of #28: prove the drop-in claim. Wire up upstream's own test scripts against our libzstd.so.1 + zstd binary, run cross-validation against C libzstd on diverse corpora, verify our .so exports the exact symbol set upstream does, and sample-build a handful of real reverse-dependencies to confirm they work without recompilation.

Why this is its own issue

The earlier Phase 6 issues land the implementation; this one lands the evidence. Without it, "drop-in replacement" is a claim — with it, the claim is testable and graphable per commit.

Deliverables

1. Upstream test-suite harness

Vendor (or git-submodule) upstream's tests/ directory at v1.5.7. Wire up CI jobs to run:

• tests/playTests.sh — upstream's main shell-driven integration suite. Hundreds of compress/decompress/stdin-pipe/dictionary/multi-frame scenarios. Runs against ${ZSTD} and ${DATAGEN}; we set ZSTD=$(target_dir)/release/zstd and DATAGEN to either upstream's datagen binary or a pure-Rust replacement.
• tests/fileTests.sh — file-handling edge cases.
• tests/playTests.sh --test-large-data — multi-GB stream tests on CI with enough disk.
• tests/dict-builder/ scripts — exercise our ZDICT_* FFI vs upstream's dictbuilder reference.

CI job structure:

.github/workflows/conformance.yml:
  - matrix: { os: [ubuntu, macos], target: [x86_64, aarch64] }
  - steps: build c-api .so + cli binary → set ZSTD env → run playTests.sh
  - failure: dump frames + first diff vs upstream output

2. Cross-validation harness

Per-corpus matrix:

• silesia/ (standard zstd benchmark corpus, vendored or downloaded in CI)
• canterbury/ (small canonical files)
• enwiki-1m/ (Wikipedia text sample)
• decodecorpus-generated synthetic (already in our test infra)

For each corpus × compression_level ∈ {1, 3, 9, 19, 22}:

• A1: our zstd encode → upstream zstd decode → byte-exact match required.
• A2: upstream zstd encode → our zstd decode → byte-exact match required.
• A3: our zstd encode → our zstd decode → byte-exact (basic roundtrip, already covered, kept as regression).
• B1: compressed_size_ours / compressed_size_upstream per (corpus, level) — tracked over time in a CI graph; ≤ 1.05 (5% slack) is the parity target.
• C1: compress_time_ours / compress_time_upstream per (corpus, level) — tracked; ≤ 1.5× is the parity target (Phase 6 perf claim).

A failure means either (i) we produced an invalid frame upstream rejects, or (ii) we rejected a valid upstream frame, or (iii) we silently corrupted output. All three are dealbreakers for drop-in.

3. ABI symbol verification

• c-api/tests/symbol_snapshot.rs: stores nm -D libstructured_zstd.so | sort output as a checked-in fixture. CI regenerates and diffs — any new export, removed export, or changed binding (e.g. T → W) breaks the build.
• Sentinel snapshot: also check that nm -D of upstream libzstd.so.1.5.7 against our .so produces an identical symbol set (modulo Rust-internal _ZN... mangled symbols which we mark with #[no_mangle] everywhere visible).
• Static struct size asserts (extending feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers) #126's abi.rs): ZSTD_inBuffer/ZSTD_outBuffer/ZSTD_frameHeader size_of matches upstream sizeof(...).

4. Reverse-dep smoke build

Vendor a handful of small real-world consumers and build them against our .so instead of upstream's:

• tar with --zstd: take tar source, LD_PRELOAD=path/to/libstructured_zstd.so.1 tar --zstd -cf out.tar.zst dir/, roundtrip extract via upstream tar --zstd → input dir.
• zstdcat | wc -c over a known archive: smoke check the CLI dispatch + stdin/stdout glue work in a real shell pipeline.
• Kernel zstd userspace tool (linux/lib/zstd/'s userspace test binary): compile against our headers, run the kernel's own unit tests against our impl.
• rsync --compress-choice=zstd (if available): rsync with zstd compression negotiated; verify the receiving side decompresses.

This isn't a packaging effort — we don't ship .apk/.deb/.rpm from CI. It's a "does it actually work when wired into a real binary that didn't know it was being swapped" smoke check.

5. Performance graph publication

Reuse the existing bench infra (#28 Phase 3+): publish compressed_size_ours / upstream and compress_time_ours / upstream per (corpus, level) on gh-pages, charted over time. Alerts fire when either ratio crosses a threshold.

Out of scope

• Building actual .apk / .deb packages — that's downstream packaging work, not us.
• Replacing system libzstd.so.1 on a developer machine via LD_PRELOAD in everyday workflows — possible from this work but not required.
• Bug-for-bug compatibility with upstream's UB / undefined-on-purpose behaviour. We match the spec, not the bugs.

Acceptance criteria

• tests/playTests.sh runs end-to-end with zero failures on our binary.
• Cross-validation matrix (corpus × level × A1/A2/A3): byte-exact roundtrip pass on every cell.
• Ratio parity gate: compressed_size_ours / upstream ≤ 1.05 for every (corpus, level) combo on the standard corpora.
• Time parity gate: compress_time_ours / upstream ≤ 1.5 on Linux x86_64 (the platform the bench-matrix targets).
• nm -D of our .so is a strict superset of upstream's exported symbol set; CI fails on regressions.
• At least 3 of the 4 reverse-dep smoke tests pass (tar --zstd, zstdcat | wc, kernel userspace tool, rsync — rsync may be marked optional if not present in CI image).
• Performance graph published on gh-pages with per-corpus charts.

Estimate

~10-12 working days (vendor + harness wiring + ABI snapshot infra + reverse-dep glue + graph integration).

Blocked by

• feat(cli): #28 Phase 6.3 — zstd CLI binary with full upstream v1.5.7 parity #128 (Phase 6.3 — CLI) — tests/playTests.sh needs a working CLI binary.
• feat(c-api): #28 Phase 6.4 — zstdmt donor protocol parity (overlapLog + rsyncable + FFI/CLI), built on #19 #129 (Phase 6.4 — MT) — playTests.sh has MT-aware scenarios that need real MT.
• feat(decoding): #28 Phase 6.6 — legacy frame format decoders v0.5-v0.7 #131 (Phase 6.6 — legacy v0.5-v0.7) — playTests.sh covers legacy archives.

References

• Upstream tests/ directory v1.5.7
• Phase 6 design: Roadmap: structured-zstd feature parity with C zstd #28
• Existing bench infra: .github/workflows/ci.yml bench-matrix jobs

[coderabbitai]

🔗 Related PRs

#38 - test(bench): expand zstd benchmark suite [merged]
#43 - test(bench): expand benchmark parity matrix [merged]
#65 - perf(bench): add rust/ffi delta benchmark artifacts [merged]
#107 - perf(bench): add wildcopy candidate research bench and dashboard range filter [merged]
#119 - refactor(encoding): #111 Phase 1e — migrate methods off HcMatchGenerator (Stages A–D) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!