Skip to content

feat: support uninitialized output buffers - #19

Merged
tisonkun merged 4 commits into
fast:mainfrom
patrickZWY:feat/uninit-output
Nov 18, 2026
Merged

tisonkun merged 4 commits into
fast:mainfrom
patrickZWY:feat/uninit-output

Conversation

@patrickZWY

@patrickZWY patrickZWY commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Add safe caller-buffer APIs that write directly into &mut [MaybeUninit<u8>]:

  • encode_into_uninit returns the initialized &mut str prefix.
  • decode_into_uninit returns the initialized &mut [u8] prefix.

These methods support uninitialized stack buffers and Vec/BytesMut spare capacity without prior zero-filling. They are available on Codec and concrete profiles. Existing signatures remain unchanged, and no dependencies are added.

Closes #11.

Design Notes

  • Choose separate methods over a unified output wrapper. The measured prototypes show no consistent performance advantage for either design. Separate methods make the storage choice explicit and preserve existing calls without requiring a wrapper.
  • Share the implementation with existing methods. Initialized slices use the same private codec path, keeping capacity checks, backend calls, and error handling together. The library returns initialized prefixes only after success. Errors may leave partial writes; container lengths remain unchanged.
  • Keep streaming update/finish APIs on initialized slices. Callers can initialize a chunk buffer once and reuse it. This keeps the API small and avoids extending state and retry behavior across four additional methods. Streaming still cannot target spare capacity directly. This is a scope and maintenance decision; new streaming variants were not benchmarked.

Performance Evidence

We compared separate methods with a zero-dependency Output wrapper using the same backend, harness, inputs, and lockfile. Positive changes mean the wrapper takes longer.

Release configuration Timed scenarios Geometric mean change Per-scenario change
Thin LTO, 1 codegen unit 9 +0.126% −0.145% to +0.573%
No LTO, 16 codegen units 6 −0.096% −0.521% to +0.169%

Environment: Intel Core i5-8279U at 2.40 GHz, x86_64, 4 cores / 8 threads, 8 GiB RAM; macOS 15.7.9; rustc 1.100.0 / LLVM 23.1.5. No extra RUSTFLAGS or CPU affinity.

Method: baseline 1b226ab; 32 samples per API per scenario, approximately 300 ms each, fixed matched round counts, alternating ABBA/BAAB blocks, and 10,000 paired bootstrap resamples. Both configurations cover Base16/Base32/Base64 encoding and decoding at 8 bytes using reused uninitialized buffers. Thin LTO adds three 16 KiB cases. Sizes refer to raw messages; timed loops do not allocate.

Interpretation: every scenario’s 95% confidence interval includes zero. One interval extends to +1.185%, so strict ±1% equivalence was not established for every case. The results support choosing on API clarity and compatibility rather than a measured speed advantage. This comparison evaluates two APIs supporting uninitialized output; it does not measure initialization savings.

Against the updated base c815982, both configurations also produce identical workload functions for all 96 matched API pairs and pass 192 output checks each. This is code-generation verification; timings were not repeated on that base. Conclusions remain specific to the tested prototypes, compiler, and machine.

Validation

  • cargo x lint, cargo x check, and nine cross-target compile checks passed. Stable and Rust 1.88.0 each passed 685 test executions; benchmark verification passed 209 checks.
  • Independent wire-byte contracts cover all 24 profiles. Three focused Miri tests, 24 uninitialized-buffer contracts under Miri, and additional AddressSanitizer/Tree Borrows checks passed. Vec/BytesMut consumer checks passed on both toolchains.
  • Existing-API verification against c815982 covered 46 scenarios and produced matching benchmark executable .text sections. Native ARM and WebAssembly execution were not performed locally.

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your information! LGTM.

@tisonkun
tisonkun merged commit 941cfd2 into fast:main Nov 18, 2026
13 checks passed
@patrickZWY
patrickZWY deleted the feat/uninit-output branch November 18, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support uninitialized caller output buffers (MaybeUninit<u8>)

2 participants