Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 12 additions & 88 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ on:
pull_request:
push:
branches: [main]
tags: ["v*"]
workflow_dispatch:

permissions:
Expand Down Expand Up @@ -122,116 +121,41 @@ jobs:
run: cargo x test

target:
name: Target (${{ matrix.name }})
name: Targets
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- name: x86_64 AVX2
target: x86_64-unknown-linux-gnu
features: --all-features
rustflags: -C target-feature=+avx2
- name: x86_64 SSSE3
target: x86_64-unknown-linux-gnu
features: --no-default-features
rustflags: -C target-feature=+ssse3
- name: AArch64 NEON
target: aarch64-unknown-linux-gnu
features: --no-default-features
rustflags: -C target-feature=+neon
- name: WebAssembly
target: wasm32-unknown-unknown
features: --no-default-features
rustflags: ""
- name: WebAssembly SIMD128
target: wasm32-unknown-unknown
features: --no-default-features
rustflags: -C target-feature=+simd128
- name: bare metal
target: thumbv7em-none-eabi
features: --no-default-features
rustflags: ""
env:
RUSTUP_TOOLCHAIN: stable
RUSTFLAGS: -D warnings ${{ matrix.rustflags }}
RUSTFLAGS: -D warnings
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install Rust target
- name: Install Rust targets
run: |
rustup toolchain install stable --profile minimal --no-self-update
rustup target add "${{ matrix.target }}"
rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu wasm32-unknown-unknown thumbv7em-none-eabi

- name: Restore Rust cache
uses: swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
with:
key: ${{ matrix.target }}

- name: Check target
run: cargo check --package bases --target "${{ matrix.target }}" --locked ${{ matrix.features }}

- name: Check no_std feature combinations
if: matrix.target == 'thumbv7em-none-eabi'
- name: Check targets and no_std features
run: |
RUSTFLAGS="-D warnings -C target-feature=+avx2" cargo check --package bases --target x86_64-unknown-linux-gnu --locked --all-features
RUSTFLAGS="-D warnings -C target-feature=+ssse3" cargo check --package bases --target x86_64-unknown-linux-gnu --locked --no-default-features
RUSTFLAGS="-D warnings -C target-feature=+neon" cargo check --package bases --target aarch64-unknown-linux-gnu --locked --no-default-features
cargo check --package bases --target wasm32-unknown-unknown --locked --no-default-features
RUSTFLAGS="-D warnings -C target-feature=+simd128" cargo check --package bases --target wasm32-unknown-unknown --locked --no-default-features
cargo check --package bases --target thumbv7em-none-eabi --locked --no-default-features
cargo check --package bases --target thumbv7em-none-eabi --locked --no-default-features --features alloc
cargo check --package bases --target thumbv7em-none-eabi --locked --no-default-features --features serde
cargo check --package bases --target thumbv7em-none-eabi --locked --no-default-features --features alloc,serde

package:
name: Package
runs-on: ubuntu-24.04
timeout-minutes: 10
env:
RUSTUP_TOOLCHAIN: stable
steps:
- name: Check out repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- name: Install Rust
run: rustup toolchain install stable --profile minimal --no-self-update

- name: Restore Rust cache
uses: swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3

- name: Validate package contents
shell: bash
run: |
package_files="$(cargo package --package bases --locked --list)"
for required in Cargo.lock Cargo.toml README.md LICENSE src/lib.rs; do
grep -Fxq "$required" <<< "$package_files" || {
echo "missing required package file: $required" >&2
exit 1
}
done
if grep -Eq '^(benchmarks|examples|tests-integration|xtask|target)/' <<< "$package_files"; then
echo "development-only files leaked into the package" >&2
exit 1
fi

- name: Build package
run: cargo package --package bases --locked

- name: Test package
shell: bash
run: |
package_dir="$(find target/package -mindepth 1 -maxdepth 1 -type d -name 'bases-*' -print -quit)"
test -n "$package_dir"
cargo test --manifest-path "$package_dir/Cargo.toml" --all-features --locked
cargo test --manifest-path "$package_dir/Cargo.toml" --no-default-features --locked
cargo test --manifest-path "$package_dir/Cargo.toml" --no-default-features --features alloc,serde --locked
cargo test --manifest-path "$package_dir/Cargo.toml" --no-default-features --features serde --locked

required:
name: Required
if: ${{ always() }}
needs: [lint, msrv, test, target, package]
needs: [lint, msrv, test, target]
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
Expand Down
16 changes: 15 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,21 @@
# CHANGELOG
# Changelog

All significant changes to this project will be documented in this file.

## v0.1.1 (2026-11-17)

### Improvements

* Include a self-contained usage guide and architecture documentation in the crate package, and organize repository navigation for examples, benchmarks, and release notes.

### Performance improvements

* Avoid redundant initialization, copying, and UTF-8 validation when encoding or decoding owned results, appending to existing buffers, and decoding fixed-size arrays.
* Reuse decoded-length and padding analysis across decoding paths to avoid repeated validation.
* Reduce formatting and incremental streaming overhead by processing complete encoding quanta directly.
* Improve scalar and SIMD handling of short inputs and tails across Base16, Base32, and Base64, including simpler x86 Base64 character mapping and Base16 decoding.
* Let `io::EncoderWriter` process larger input chunks according to the encoding profile, and avoid redundant buffer initialization across short reads in the streaming example.

## v0.1.0 (2026-11-15)

This is the initial release.
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ resolver = "3"
[workspace.package]
edition = "2024"
license = "Apache-2.0"
readme = "README.md"
repository = "https://github.com/fast/bases"
rust-version = "1.88.0"

Expand Down
77 changes: 21 additions & 56 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# bases
# Bases

[![Crates.io][crates-badge]][crates-url]
[![Documentation][docs-badge]][docs-url]
Expand All @@ -18,7 +18,7 @@

Fast, strict RFC 4648 Base16, Base32, and Base64 codecs for Rust.

Bases provides configurable zero-sized family types (`base16::Base16`, `base32::Base32`, `base64::Base64`) parameterized by `policy` markers alongside ten crate-root profile aliases (`Base16Lower`, `Base16Upper`, `Base32`, `Base32Unpadded`, `Base32Hex`, `Base32HexUnpadded`, `Base64`, `Base64Unpadded`, `Base64Url`, `Base64UrlUnpadded`). Every profile exposes direct inherent methods (including `const fn` sizing and constructors) without requiring trait imports, and implements the sealed `Codec` trait for generic callers.
Bases supports allocation-free caller buffers, owned results, incremental streaming, formatting, and Serde integration. It works in `no_std` without dependencies by default.

```rust
use bases::{Base64, Base64UrlUnpadded};
Expand All @@ -31,71 +31,36 @@ let encoded = Base64UrlUnpadded::encode_into(b"hello", &mut buffer).unwrap();
assert_eq!(encoded, "aGVsbG8");
```

## API layers
## Documentation

- `encoded_len`, `max_decoded_len`, `encode_into`, `decode_into`, `decode_in_place`, and `decode_array` provide allocation-free operations, with `const fn` support on concrete profile `name`, `encoded_len`, `max_decoded_len`, `display`, `encoder`, and `decoder`.
- `encode`, `encode_append`, `decode`, and `decode_append` provide `String` and `Vec<u8>` helpers when the `alloc` feature is enabled.
- `Encoder<C>` and `Decoder<C>` process arbitrarily chunked input while retaining at most one partial quantum; the `std` feature adds `bases::io::{EncoderWriter, DecoderReader}` for `std::io::{Write, Read}`.
- `display` (`EncodedDisplay`) and `HexDisplay` (`LowerHex` `{:x}` and `UpperHex` `{:X}` only) write encoded data directly through `core::fmt`.
- `serialize`, `deserialize_array`, and (with `alloc`) `deserialize` integrate with `#[serde(with = "...")]` when the `serde` feature is enabled.
Add `bases = "0.1"` to your dependencies. The [usage guide](bases/README.md) covers profiles, feature flags, streaming, and error handling.

```rust
use bases::Base64;

let mut encoder = Base64::encoder();
let mut output = [0_u8; 4];
assert_eq!(encoder.update(b"fo", &mut output).unwrap(), "");
assert_eq!(encoder.update(b"o", &mut output).unwrap(), "Zm9v");
assert_eq!(encoder.finish(&mut output).unwrap(), "");
```

Bases is `#![no_std]` by default (`default = []`), exposing caller-buffer, in-place, fixed-array, streaming, and formatting APIs with zero default features. The `alloc` feature adds `String` and `Vec<u8>` helpers; `std` enables `alloc`, `bases::io::{EncoderWriter, DecoderReader}`, and runtime x86/x86_64 CPU feature detection; `serde` adds `serde_core` serialization and fixed-array deserialization in `no_std`, plus `Vec<u8>` deserialization when combined with `alloc`.

Successful `finish()` consumes the stream. On failure, `FinishError::into_inner()` recovers the unchanged encoder or decoder for retrying with a larger output buffer. Callers that do not need recovery can use `?` to convert `FinishError` into the underlying `EncodeError`, `DecodeError`, or `DecodeSliceError`, discarding the stream.

## Examples

Runnable examples live in the `examples` workspace crate. Each program uses local sample data and can run without arguments from the repository root:

```shell
cargo run --package examples --example url_cursor
```
| Task | Read |
| ----------------------------- | ----------------------------------------- |
| Choose a profile or API | [Usage guide](bases/README.md) |
| Look up public contracts | [API reference](https://docs.rs/bases) |
| Run application examples | [Examples](examples/README.md) |
| Understand the implementation | [Architecture](bases/ARCHITECTURE.md) |
| Measure a workload | [Benchmarks](benchmarks/README.md) |
| Review release changes | [Changelog](CHANGELOG.md) |

| Example | Application | APIs to start with |
|--------------------------------------------------------|---------------------------------------------------------------------------------------------|----------------------------------------------------------|
| [`base64_field`](examples/src/base64_field.rs) | Carry binary data in a JSON string field and reject a malformed field. | `Base64::encode`, `Base64::decode`, `DecodeError` |
| [`url_cursor`](examples/src/url_cursor.rs) | Encode a record ID into an unpadded URL-safe pagination cursor and recover it. | `Base64UrlUnpadded::encode`, `decode_array` |
| [`reusable_buffers`](examples/src/reusable_buffers.rs) | Reuse output buffers across a batch of records with different encoding profiles. | `C: Codec`, sizing helpers, `encode_into`, `decode_into` |
| [`stream_io`](examples/src/stream_io.rs) | Convert a file or network stream with bounded buffers and explicit finalization. | `Encoder<C>`, `Decoder<C>`, `update`, `finish` |
| [`format_bytes`](examples/src/format_bytes.rs) | Format binary IDs and payloads in diagnostic output without an intermediate encoded string. | `HexDisplay` (`{:x}`/`{:X}`), `Base64::display` |
| [`profiles`](examples/src/profiles.rs) | Compare Base16, Base32, Base64, and Base64URL representations. | `Base16Lower`, `Base16Upper`, `Base32`, `Base64` |
## Development

The `stream_io` example also accepts `encode` or `decode` to transform standard input into standard output using padded Base64:
The workspace separates the library, examples, integration tests, benchmarks, and repository tooling. Run checks from the repository root:

```shell
printf 'hello, world' | cargo run --quiet --package examples --example stream_io -- encode
printf 'aGVsbG8sIHdvcmxk' | cargo run --quiet --package examples --example stream_io -- decode
cargo x check
cargo x test
cargo x lint
```

Its `encode` and `decode` helpers accept any `Read` and `Write`, including files and sockets. Successful updates consume the complete input chunk; `finish()` handles the final fragment at EOF. A decoding error can arrive after earlier output has been written, so callers should treat the destination as complete only after the whole operation succeeds.

## Strictness

Decoders reject whitespace, bytes outside the selected alphabet, misplaced or forbidden padding, and non-zero unused trailing bits. Padded and unpadded protocols use different profile types; `Base16Lower` and `Base16Upper` default to RFC 4648 case-insensitive decoding (`policy::Mixed`), while strict single-case Base16 and case-insensitive Base32 (such as RFC 5155 `Base32<ExtendedHex, Unpadded, Lower, Mixed>`) can be selected explicitly via `policy` markers.

Base encoding is not encryption. Bases does not provide constant-time guarantees and uses data-dependent branches, table lookups, and private scalar or SIMD backends (`aarch64` NEON, `x86`/`x86_64` SSSE3 and AVX2, and `wasm32` `simd128`).

## Architecture

[Architecture](ARCHITECTURE.md) records the public layers, private backend boundary, and performance contracts. See [CHANGELOG.md](CHANGELOG.md) for release notes.

## Benchmarks
Use `cargo x bench` for performance comparisons and see the [benchmark guide](benchmarks/README.md) for workloads and measurement limits. See [Releasing Bases](RELEASE.md) for release validation and publication.

Run `cargo x bench` to compare caller buffers, owned results, formatting, streaming, and invalid-input handling against other Rust codecs. The [benchmark guide](benchmarks/README.md) lists the workloads and explains how to read the timings.
## Minimum Supported Rust Version (MSRV)

## Minimum Rust version policy
This crate is built against the latest stable release, and its minimum supported rustc version is 1.88.0.

The minimum supported Rust version is 1.88.0. It may be increased in a minor release, but not in a patch release.
The policy is that the minimum Rust version required to use this crate can be increased in minor version updates. For example, if Bases 1.0 requires Rust 1.20.0, then Bases 1.0.z for all values of z will also require Rust 1.20.0 or newer. However, Bases 1.y for y > 0 may require a newer minimum version of Rust.

## License

Expand Down
8 changes: 4 additions & 4 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# Releasing Bases

Releases are published manually to crates.io. For the first release, check that the crate name is still available and the repository linked in the package metadata is accessible to its intended readers.
Releases are published manually to crates.io from a clean commit that has passed CI.

## Prepare

1. Update the version in `bases/Cargo.toml` and refresh `Cargo.lock`.
2. Move user-visible entries from `Unreleased` in `CHANGELOG.md` into a versioned section.
3. Run `cargo x lint`, `cargo x check`, `cargo x test`, and `cargo package --package bases --locked`.
4. Inspect the archive with `cargo package --package bases --locked --list`. Run `cargo publish --dry-run --package bases --locked` from the clean release commit to verify the package without uploading it.
2. Add a versioned section to the [changelog](CHANGELOG.md) for user-visible changes.
3. Run `cargo x lint`, `cargo x check`, and `cargo x test`.
4. From the clean release commit, review `cargo package --list --package bases --locked` and run `cargo publish --dry-run --package bases --all-features --locked`. Cargo builds the archive and verifies that the extracted crate compiles without uploading it. Check that the package contains its README, architecture guide, and license, with relative documentation links resolving inside the package.

## Publish

Expand Down
6 changes: 5 additions & 1 deletion ARCHITECTURE.md → bases/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,14 @@ Each profile maps at compile time to private backend dispatch functions that pai
- `x86` and `x86_64` AVX2 and SSSE3 for Base16, Base32, and Base64, using runtime `is_x86_feature_detected!` dispatch under `std` or compile-time `target_feature` selection in `no_std`.
- `wasm32` `simd128` (`target_feature = "simd128"`) for Base16 and Base64.

SIMD backends process aligned bulk prefixes and fall back to the scalar reference implementations for tails and under `cfg!(miri)`. Callers cannot select or observe a backend, and every SIMD kernel preserves exact scalar output, error precedence, error byte offsets, and stream-state contracts. While the workspace denies `unsafe_code`, private `backend/*/arch` modules carry scoped `#[allow(unsafe_code)]` with `missing_safety_doc` and `undocumented_unsafe_blocks` enforced.
SIMD backends process bulk prefixes and fall back to the scalar reference implementations for tails and under `cfg!(miri)`. Backend selection is private, and every SIMD kernel preserves exact scalar output, error precedence, error byte offsets, and stream-state contracts.

The workspace denies `unsafe_code` by default. Scoped allowances cover architecture intrinsics and internal buffer operations, including writing into uninitialized storage and converting initialized ASCII bytes to strings. These operations document their safety invariants, with `missing_safety_doc` and `undocumented_unsafe_blocks` enforced.

## Performance contract

The primary API writes into caller-owned slices, arrays, or in-place mutable buffers without allocating. Sizing methods let callers reuse exact encoding buffers and safe upper-bound decoding buffers. Successful stream updates consume the complete input slice; failures leave stream state unchanged so callers can retry with more output capacity.

Owned, appended, and fixed-array results are written directly into their destination storage. Only initialized output is exposed to callers; spare capacity does not require zeroing or a second copy. Formatting encodes complete input quanta into a bounded scratch buffer before writing to the formatter.

In-place decoding (`decode_in_place`) writes the decoded prefix back over the encoded buffer and leaves buffer contents unspecified on `DecodeError`. Constant-time execution and in-place encoding are not provided.
4 changes: 2 additions & 2 deletions bases/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,14 @@
[package]
description = "Fast, strict RFC 4648 Base16, Base32, and Base64 codecs for Rust."
name = "bases"
version = "0.1.0"
readme = "README.md"
version = "0.1.1"

categories = ["encoding", "no-std"]
documentation = "https://docs.rs/bases"
edition.workspace = true
keywords = ["base16", "base32", "base64", "encoding", "no-std"]
license.workspace = true
readme.workspace = true
repository.workspace = true
rust-version.workspace = true

Expand Down
Loading
Loading