feat(c-api): #28 Phase 6.2 — advanced + streaming + dictionary C FFI surface

[polaz]

Progress

Landed via the C ABI core + CDict work (PRs #386, #387): ZSTD_createCDict / ZSTD_createCDict_byReference / ZSTD_freeCDict / ZSTD_sizeof_CDict / ZSTD_compress_usingCDict / ZSTD_getDictID_fromCDict, the full DDict mirror, and the simple/context/frame/error surface (39 symbols total). Remaining in this issue: the advanced parameter API (ZSTD_CCtx_setParameter family, ZSTD_compress2, ZSTD_compressStream2, estimates), the streaming ZSTD_CStream/ZSTD_DStream surface, and the _advanced dictionary variants + *_usingDict one-shots.

Summary

Phase 6.2 of #28: complete the v1.5.7 advanced/streaming/dictionary C API surface on top of the FFI foundation from #126 (Phase 6.1). After this issue lands, libzstd.so.1 produced by c-api/ is symbol-complete vs upstream v1.5.7 except for multi-threaded compression (Phase 6.4 / #64) and legacy decoders (Phase 6.5/6.6 / #65/#66).

Deliverables

1. Advanced parameter API (ZSTD_CCtx_setParameter family)

Requires #27 (configurable parameters API) to be merged first — that issue exposes the Rust-side parameter struct this FFI layer wraps.

Wrappers:

• ZSTD_CCtx_setParameter, ZSTD_CCtx_getParameter, ZSTD_CCtx_setPledgedSrcSize
• ZSTD_DCtx_setParameter, ZSTD_DCtx_getParameter
• ZSTD_cParam_getBounds, ZSTD_dParam_getBounds
• ZSTD_CCtx_reset, ZSTD_DCtx_reset
• ZSTD_compress2, ZSTD_compressStream2
• ZSTD_estimateCCtxSize, ZSTD_estimateDCtxSize, ZSTD_estimateCCtxSize_usingCParams, ZSTD_estimateCStreamSize, ZSTD_estimateDStreamSize

Parameter enum (ZSTD_cParameter, ZSTD_dParameter) covers all v1.5.7 entries:

• ZSTD_c_compressionLevel, ZSTD_c_windowLog, ZSTD_c_hashLog, ZSTD_c_chainLog, ZSTD_c_searchLog, ZSTD_c_minMatch, ZSTD_c_targetLength, ZSTD_c_strategy
• ZSTD_c_enableLongDistanceMatching, ZSTD_c_ldmHashLog, ZSTD_c_ldmMinMatch, ZSTD_c_ldmBucketSizeLog, ZSTD_c_ldmHashRateLog
• ZSTD_c_contentSizeFlag, ZSTD_c_checksumFlag, ZSTD_c_dictIDFlag
• ZSTD_c_jobSize, ZSTD_c_overlapLog, ZSTD_c_rsyncable
• ZSTD_c_nbWorkers — accepts only 0 in this issue; non-zero handled by 6.4 (chore: release v0.0.7 #64)
• ZSTD_c_useRowMatchFinder, ZSTD_c_targetCBlockSize (already implemented via perf(encoding): implement row-based match finder for fast/dfast levels #67 / feat: block splitting for improved compression ratio #23 internals)

2. Streaming API

ZSTD_CStream / ZSTD_DStream are aliases for ZSTD_CCtx / ZSTD_DCtx per v1.5.x design. The streaming functions ride on top of the same opaque contexts.

Wrappers:

• ZSTD_createCStream, ZSTD_freeCStream, ZSTD_initCStream, ZSTD_compressStream, ZSTD_flushStream, ZSTD_endStream, ZSTD_CStreamInSize, ZSTD_CStreamOutSize
• ZSTD_createDStream, ZSTD_freeDStream, ZSTD_initDStream, ZSTD_decompressStream, ZSTD_DStreamInSize, ZSTD_DStreamOutSize
• ZSTD_inBuffer / ZSTD_outBuffer structs are #[repr(C)] aliases of our streaming buffer types.

3. Dictionary API

Compression-side (ZSTD_CDict):

• ZSTD_createCDict, ZSTD_createCDict_byReference, ZSTD_createCDict_advanced
• ZSTD_freeCDict, ZSTD_sizeof_CDict
• ZSTD_compress_usingCDict, ZSTD_compress_usingCDict_advanced
• ZSTD_CCtx_loadDictionary, ZSTD_CCtx_loadDictionary_byReference, ZSTD_CCtx_loadDictionary_advanced
• ZSTD_CCtx_refCDict, ZSTD_CCtx_refPrefix, ZSTD_CCtx_refPrefix_advanced

Decompression-side (ZSTD_DDict):

• ZSTD_createDDict, ZSTD_createDDict_byReference, ZSTD_createDDict_advanced
• ZSTD_freeDDict, ZSTD_sizeof_DDict
• ZSTD_decompress_usingDDict
• ZSTD_DCtx_loadDictionary, ZSTD_DCtx_refDDict, ZSTD_DCtx_refPrefix
• ZSTD_getDictID_fromCDict, ZSTD_getDictID_fromDDict, ZSTD_getDictID_fromDict, ZSTD_getDictID_fromFrame

zdict.h training API:

• ZDICT_trainFromBuffer
• ZDICT_optimizeTrainFromBuffer_fastCover
• ZDICT_trainFromBuffer_fastCover
• ZDICT_finalizeDictionary
• ZDICT_isError, ZDICT_getErrorName
• ZDICT_getDictHeaderSize, ZDICT_getDictID

4. ABI invariants

• ZSTD_inBuffer / ZSTD_outBuffer size_of snapshots match upstream sizeof(ZSTD_inBuffer) (= 24 bytes on 64-bit per upstream zstd.h).
• ZSTD_cParameter / ZSTD_dParameter enum discriminants match upstream numeric values byte-for-byte (asserted via the vendored header diff in feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers) #126 plus static_assertions in abi.rs).
• Streaming ZSTD_CStreamInSize() / ZSTD_CStreamOutSize() return the same values upstream does (matches ZSTD_BLOCKSIZE_MAX).

Out of scope

• Multi-threaded ZSTD_c_nbWorkers > 0 → 6.4 (chore: release v0.0.7 #64). This issue treats nbWorkers > 0 as a graceful no-op (single-threaded fallback) with a deprecation warning, so consumers that set it don't crash.
• CLI binary → 6.3 (feat(encoding): numeric compression levels (1-22) API #63).
• Legacy frames → 6.5 (perf(bench): add rust/ffi delta benchmark artifacts #65) / 6.6 (perf(decoding): SIMD HUF decode kernels with runtime CPU dispatch #66).

Acceptance criteria

• All listed wrappers compile + link + are exercised by a C consumer test (extends c_consumer.c from feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers) #126 to drive streaming + advanced + dictionary paths).
• ZSTD_CCtx_setParameter accepts every v1.5.7 ZSTD_cParameter discriminant; rejects out-of-bounds values via ZSTD_cParam_getBounds.
• Streaming round-trip via ZSTD_compressStream2 + ZSTD_decompressStream produces byte-exact output on a 16 MiB random + low-entropy + high-entropy corpus.
• Dictionary round-trip via ZSTD_compress_usingCDict + ZSTD_decompress_usingDDict matches the same Rust-API roundtrip byte-for-byte.
• Dictionary training via ZDICT_trainFromBuffer produces a dictionary that a C consumer can use with ZSTD_compress_usingDict, decode-side via our .so succeeds.
• Symbol snapshot from feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers) #126 (c-api/tests/symbols.rs) updated to cover the full v1.5.7 surface.
• nbWorkers = N > 0 returns gracefully (current FFI: ignored with warning), to be replaced by real MT in 6.4 (chore: release v0.0.7 #64).

Estimate

~10-12 working days (~60 wrappers including ABI tests).

Blocked by

• feat(c-api): #28 Phase 6.1 — C ABI core (cdylib + vendored headers + simple/context/error wrappers) #126 (Phase 6.1) — needs the workspace + vendored headers + error mapping infra.
• feat: configurable compression parameters API #27 (configurable parameters API) — ZSTD_CCtx_setParameter is a thin shim on top of feat: configurable compression parameters API #27's CompressionParameters builder.

References

• Upstream zstd.h v1.5.7 advanced API section
• Upstream zdict.h v1.5.7
• Phase 6 design: Roadmap: structured-zstd feature parity with C zstd #28

[coderabbitai]

🔗 Related PRs

#54 - perf(decoding): 4-stream interleaved Huffman decode and bulk table init [merged]
#236 - test(bench): add decompress-dict group to compare_ffi [merged]
#277 - test(bench): unblock dict-driven bench matrix + add pure_rust_with_dict compress arm [merged]
#360 - feat(encode): configurable compression parameters API [merged]
#386 - feat(c-api): C ABI core (cdylib + vendored headers + simple/context/error/frame/dict wrappers) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[polaz]

Scope clarification after #129 closure (roadmap update 2026-05-15): the ZSTD_c_nbWorkers > 0 path lands inline in this issue, wired through #19's rayon-based MT pool. Producing a valid multi-frame zstd archive via #19 satisfies the FFI contract for any consumer linking against libzstd.so.1. Donor-zstdmt byte-parity refinements (overlapLog / rsyncable / etc.) are explicitly out of scope; the prereq list now reads #126, #27, #19. No separate Phase 6.4 issue.