Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions bases/src/backend/base32.rs
Original file line number Diff line number Diff line change
Expand Up @@ -943,6 +943,31 @@ mod tests {
check_roundtrip_and_arch_parity::<true, false, false, false>();
}

#[test]
fn scalar_bulk_reports_global_error_offsets() {
// 32 complete quanta followed by the RFC 4648 encoding of "f".
let mut input = [b'A'; 264];
input[256..].copy_from_slice(b"MY======");
let plan = decode_plan::<true>(&input).unwrap();
let mut output = [MaybeUninit::uninit(); 161];

input[255] = b'!';
assert_eq!(
decode_scalar_bulk::<false, true, true>(&input, plan, &mut output),
Err(DecodeError::InvalidByte {
index: 255,
byte: b'!',
})
);

input[255] = b'A';
input[257] = b'Z';
assert_eq!(
decode_scalar_bulk::<false, true, true>(&input, plan, &mut output),
Err(DecodeError::NonCanonical { index: 257 })
);
}

fn verify_all_bytes_at_offsets<
const EXTENDED_HEX: bool,
const PADDED: bool,
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Run the same fixed workload for every implementation in a scenario. The results
| Streaming Base64 | 16 KiB | Process chunks of at most 257 bytes into a fixed destination. | `base64` I/O adapters with general-purpose and SIMD engines |
| Invalid Base64 input | 1 KiB before encoding | Reject `!` at the first or last data symbol of a padded message. | `data-encoding`, `base64`, `base64::Simd`, `base64-simd` |

All families have caller-buffer cases at 8 B, 32 B, 1 KiB and 16 KiB; Base64 also includes 64 B. This separates short-field overhead from bulk throughput. Owned workloads cover a 32-byte hex field, 1 KiB of padded Base32 or Base64, and an 8-byte unpadded URL-safe cursor. Owned decoders receive a string. Streaming includes finalization or reading to EOF, including the I/O adapter's buffering. The invalid-input cases check acceptance of the original message before corrupting one symbol.
All families have caller-buffer cases at 8 B, 32 B, 1 KiB and 16 KiB; Base64 also includes 64 B. Base32 adds 159 B, 160 B and 164 B to cover the scalar bulk threshold and a partial quantum above it. These cases use normal CPU dispatch, so exercising the bulk scalar path requires a scalar build. This separates short-field overhead from bulk throughput. Owned workloads cover a 32-byte hex field, 1 KiB of padded Base32 or Base64, and an 8-byte unpadded URL-safe cursor. Owned decoders receive a string. Streaming includes finalization or reading to EOF, including the I/O adapter's buffering. The invalid-input cases check acceptance of the original message before corrupting one symbol.

```shell
cargo x bench --no-run
Expand Down
3 changes: 3 additions & 0 deletions benchmarks/benches/throughput/buffers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ pub fn register(runner: &mut Runner) {
&[
(8, 20_000_000, 20_000_000),
(32, 15_000_000, 10_000_000),
(159, 10_000_000, 5_000_000),
(160, 10_000_000, 5_000_000),
(164, 10_000_000, 5_000_000),
(1024, 4_300_000, 2_450_000),
(16_384, 250_000, 150_000),
],
Expand Down
Loading