Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .changeset/add-png-cobs-streams.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
"@hsblabs/web-stream-extras": minor
---

Add PNG text chunk and COBS stream APIs

- `createPNGTextChunkWriter` — embed arbitrary binary payloads into PNG tEXt chunks as a transform stream
- `extractPNGTextChunk` — extract a payload from a PNG tEXt chunk (all-at-once)
- `streamPNGTextChunk` — stream-extract a payload from a PNG tEXt chunk segment by segment
- `encodeCOBSFrame` / `decodeCOBSFrame` — frame-level COBS encode/decode
- `createCOBSEncoderStream` / `createCOBSDecoderStream` — COBS transform streams
- `writeCOBS` / `readCOBS` — high-level COBS pipe helpers
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,5 @@ dist-ssr
# tmp
LESSONS.md
TODO.md
docs/superpowers/
.context/
1 change: 1 addition & 0 deletions .npmignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
.serena/
.vscode/
example/
examples/
packages/
src/
bench/
Expand Down
81 changes: 81 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,11 @@ Use this package when you need to:
- build or consume `ReadableStream<Uint8Array>` pipelines
- collect byte streams into a single `Uint8Array`
- convert between strings and `Uint8Array`
- encode or decode delimiter-separated COBS frames
- encode and decode base64 or base64url streams
- decode, encode, split, and join text streams
- encrypt or decrypt byte streams with the Web Streams API
- embed or extract binary payloads inside PNG `tEXt` chunks

This makes it a good fit for:

Expand Down Expand Up @@ -87,14 +89,57 @@ const result = await readAllBytes(decrypted);
console.log(new TextDecoder().decode(result)); // "secret payload"
```

### Encode and decode COBS frames

```ts
import {
createCOBSDecoderStream,
createCOBSEncoderStream,
} from "@hsblabs/web-stream-extras/cobs";
import { readableFromChunks } from "@hsblabs/web-stream-extras";

const rawFrames = readableFromChunks([
Uint8Array.of(0x11, 0x22, 0x00, 0x33),
Uint8Array.of(0xff, 0xee),
]);

const encoded = rawFrames.pipeThrough(createCOBSEncoderStream());
const decoded = encoded.pipeThrough(createCOBSDecoderStream());
```

### Embed and extract a binary payload in PNG

```ts
import {
createPNGTextChunkWriter,
extractPNGTextChunk,
streamPNGTextChunk,
} from "@hsblabs/web-stream-extras/png";
import { readAllBytes, readableFromChunks } from "@hsblabs/web-stream-extras";

const sourcePNG = readableFromChunks(pngBytes);
const payloadWriter = createPNGTextChunkWriter(sourcePNG);

await readableFromChunks(fileBytes).pipeTo(payloadWriter.writable);
const rebuiltPNG = await readAllBytes(payloadWriter.readable);

const extracted = await readAllBytes(
extractPNGTextChunk(readableFromChunks(rebuiltPNG)),
);

const streamed = streamPNGTextChunk(readableFromChunks(rebuiltPNG));
```

## Why Use It

This package focuses on a small set of utilities that are useful in real byte-stream pipelines:

- `readableFromChunks()` for quickly creating a `ReadableStream`
- `readAllChunks()` and `readAllBytes()` for consuming a stream
- binary conversion helpers for strings and random byte generation
- `cobs` helpers for delimiter-separated frame encoding and decoding
- `encryption` helpers for stream encryption without changing your stream-first API style
- `png` helpers for embedding and extracting binary payloads in PNG metadata

The goal is to keep Web Streams code simple, predictable, and easy to compose.

Expand Down Expand Up @@ -132,6 +177,36 @@ The `encryption` subpath provides stream encryption utilities for binary streams

`webCryptoStream(masterKey)` is a higher-level helper for applications that manage stream keys with the Web Crypto API. It uses an `AES-GCM` master key to create encrypted 32-byte stream keys, then unwraps those keys before delegating to `encryptStream()` and `decryptStream()`.

### `@hsblabs/web-stream-extras/cobs`

The `cobs` subpath provides Consistent Overhead Byte Stuffing helpers:

- `encodeCOBSFrame`
- `decodeCOBSFrame`
- `createCOBSEncoderStream`
- `createCOBSDecoderStream`
- `readCOBS`
- `writeCOBS`

`encodeCOBSFrame()` and `decodeCOBSFrame()` work on a single frame without the trailing delimiter.

The stream helpers use `0x00` as the frame delimiter. Each input chunk to `createCOBSEncoderStream()` is treated as one raw frame, and `createCOBSDecoderStream()` emits one decoded frame for each delimiter-terminated encoded frame.

### `@hsblabs/web-stream-extras/png`

The `png` subpath provides binary payload helpers for PNG files:

- `createPNGTextChunkWriter`
- `extractPNGTextChunk`
- `streamPNGTextChunk`

`createPNGTextChunkWriter()` accepts a source PNG stream and returns a `{ writable, readable }` pair. Write arbitrary `Uint8Array` payload bytes into `writable`, then read the rebuilt PNG from `readable`.

`createPNGTextChunkWriter()` forwards validated source chunks as they are read, keeps `IEND` pending, then appends internal payload chunks and the final `IEND` after `writable` closes.

`extractPNGTextChunk()` reads a PNG stream, validates the embedded payload, and then returns it as `ReadableStream<Uint8Array>`. The extractor keeps all-or-nothing semantics rather than emitting payload bytes before the final manifest and CRC are checked.

`streamPNGTextChunk()` is the late-error variant. It emits payload data segments as they arrive and only rejects at the end if the terminal manifest or payload CRC is invalid.
### `@hsblabs/web-stream-extras/text`

The `text` subpath provides small convenience wrappers for text-oriented streams:
Expand Down Expand Up @@ -210,11 +285,17 @@ console.log(new TextDecoder().decode(result)); // "secret payload"
- Browsers: works in modern browsers with Web Streams support
- Runtime APIs:
- root utilities depend on the WHATWG Streams API
- `cobs` depends on the WHATWG Streams API
- `encryption` depends on both the Web Streams API and Web Crypto
- `png` depends on the WHATWG Streams API

## Notes

- `cobs` is intentionally a subpath export. The root package stays focused on generic stream helpers.
- `encryption` is intentionally a subpath export. The root package is not encryption-only.
- `png` is intentionally a subpath export. The root package stays focused on generic stream helpers.
- This package does not handle authentication, password-based key derivation, user management, or key storage.
- For `encryption`, you are expected to provide the raw encryption key (`Uint8Array`) yourself.
- `webCryptoStream()` is optional. It is a convenience wrapper when you already manage a separate `AES-GCM` master key and want encrypted per-stream keys as strings.
- `png` stores payload bytes in internal `tEXt` chunks and does not expose low-level PNG metadata knobs in the public API.
- `png` stores payload bytes as multiple internal data segments plus one terminal manifest segment before `IEND`.
Loading