Repository navigation
Web chunk and event signers recover after a refused call and copy the buffer at call time - #74
Conversation
… buffer at call time On the WebCrypto entry, a sign call that rejected (a NaN `now`, a null message) left the internal promise queue rejected, so every later call on that signer rejected too. The Node, Bun and Deno entries throw once and stay usable. Each call now takes its turn through a queue that survives a rejection, and the chain only moves when a step succeeds. chunk() and sign() also read the caller's buffer when their turn came, not when they were called, so a caller that reused its buffer got a different signature than on the synchronous entries. They now copy at call time.
| * anything that is not a typed array, as hashing it would. | ||
| * @param {Uint8Array} data | ||
| */ | ||
| const copyOf = (data) => Uint8Array.prototype.slice.call(data); |
There was a problem hiding this comment.
medium: Copy bytes without invoking the input's species constructor · ref b8639c
copyOf uses Uint8Array.prototype.slice.call(data), which constructs its result through the input's Symbol.species.
Failure: A valid Uint8Array subclass whose species is Uint16Array produces a 16-bit copy. The signer hashes six bytes for input elements [1, 2, 255] but frames three bytes, so the service rejects the signature. The same input worked on the parent revision and synchronous entry.
Suggested fix: Copy validated input into a fresh ordinary Uint8Array without using the input's species constructor.
| return inTurn(async () => { | ||
| const { datetime, scope, date } = eventScope(now, seed.region, seed.service); | ||
| const header = dateHeader(now); | ||
| previous = await signWith( |
There was a problem hiding this comment.
medium: Commit the chain only after framing succeeds · ref f43d43
The event step assigns previous = await signWith(...) before return frameEvent(header, previous, own), while inOrder permits the next step to run after a rejection.
Failure: If framing throws after signing, such as from a recoverable output-allocation failure, the call rejects but previous already contains the signature of a frame the caller never received. A later event is accepted locally but is chained to that missing frame, so the service rejects it. The chunk path also commits before frameChunk succeeds.
Suggested fix: Build the frame using a temporary signature, then update previous only after framing succeeds; keep the complete operation inside the queue for chunks and trailers as well.
| return { | ||
| chunk: async (data) => (data.length === 0 ? data : frameChunk(await sign(data), data)), | ||
| async chunk(data) { | ||
| if (data.length === 0) { |
There was a problem hiding this comment.
medium: Keep an empty chunk empty when its buffer grows · ref d4c1ed
The empty-input branch returns the caller's view directly: if (data.length === 0) { return data; }.
Failure: If a length-tracking Uint8Array is empty when passed to chunk(), then its resizable buffer grows and is filled before the promise is awaited, the result contains the new raw bytes instead of zero bytes. Forwarding them corrupts the chunked stream, despite the new call-time buffer-reuse guarantee.
Suggested fix: Return a stable zero-length result for empty views whose backing buffer can grow, and apply the same guarantee to the synchronous entry.
| if (data.length === 0) { | ||
| return data; | ||
| } | ||
| const own = copyOf(data); // the signature and the frame are made when the chunk's turn comes; the caller may reuse its buffer once this returns |
There was a problem hiding this comment.
medium: Measure the new copy cost before merging · ref 7d2d4e
chunk() now copies each input before queueing it. CONTRIBUTING.md requires a bench/ measurement for a change that makes a workload slower, and the pull request says no benchmark was run.
Failure: In a local reproduction, queueing eight calls with the same unchanged 4 MiB buffer allocated 32 MiB of snapshots immediately; the parent allocated no extra snapshots. This can exhaust a memory-limited Worker, and the required before-and-after cost evidence is missing.
Suggested fix: Measure WebCrypto chunk and event throughput and memory for representative sizes under the repository's benchmark protocol, then record the tradeoff before merging.
|
|
||
| `chunk(data)` with an empty `data` returns it unchanged (zero bytes) and signs nothing, because a zero-length chunk ends the upload; call `end()` for that. Trailer names are lowercased before they are signed. | ||
|
|
||
| A call that is refused (a trailer with a line break, a `now` that is not a time, a message that is not a `Uint8Array`) throws, or on the promise-returning entry rejects, and leaves the signer where it was: the next call is signed with the signature before the refused one. The bytes you pass are read when you call, so you can reuse the buffer straight away. |
There was a problem hiding this comment.
medium: Honor the new promise to refuse non-Uint8Array messages · ref f00c4b
The guide lists a message that is not a Uint8Array as a refused call, but the new copy uses the generic typed-array slice method and accepts other typed-array types.
Failure: Passing new Uint16Array([0x0102]) to either event signer succeeds: hashing reads two bytes while framing writes one byte, yielding a signature the service rejects instead of the documented error.
Suggested fix: Validate the message type before signing on all entries, or narrow the new refusal promise to the inputs actually rejected and state that callers must supply a Uint8Array.
What was wrong
On the WebCrypto entry (
src/web.js, used in Workers and browsers):eventSigner.signwith aNaNnowor anullmessage rejected, and because the internal promise chain (turn = turn.then(...)) kept the rejection, every latersignon that signer rejected too. The Node, Bun and Deno entries throw once and stay usable. The same shape existed inchunkSigner, but a refusedchunkcould not be triggered by an ordinary input before this change (see Tests).chunk()hashed and frameddatawhen its turn in the queue ran, not when it was called. A caller that reused its buffer straight after the call got a signature for the new bytes, where the synchronous entries sign what was passed.eventSigner.signhad the same late read.Fix
inOrder()runs each signing step after the one before it. A step that throws rejects only its own call (turn = result.catch(() => {})), and the chain value (previous) is still assigned only after a step succeeds, so a refused call leaves the chain where it was. The existing guarantee that a refused trailer inend()leaves the chain unchanged still holds.chunk()andsign()takeUint8Array.prototype.slice.call(data)before queueing. That copies for a NodeBuffertoo (Buffer#slicereturns a view) and throws for anything that is not a typed array, as hashing it did. Cost: one extra copy of each chunk or message on this entry;frameChunkalready copies into the output, so this is a second copy of the bytes. I chose the copy over documenting "don't touch the buffer until the promise settles" so the web entry gives the same result as the others for the same calls. No benchmark was run; per AGENTS.md the speed, memory and size figures are re-measured at the next version bump, andsrc/changed.eventSigner.signgot the same copy; the report named onlychunk(), but the cause and fix are the same.Tests
test/hardening.test.jsruns each new test on the sync, dist and web entries; the web ones fail onmainand pass here (confirmed before the fix: 4 red on web, green on sync and dist):nowofNaN, and anullmessage) leaves the event signer usable and the next signature equal to a fresh signer's, including a call made straight after without waiting;Limit: after the copy, a refused
chunkthrows before it is queued, so only the event tests exercise the queue's recovery (confirmed by removing the.catch: the event test fails).bun run checkpasses in full (format, lint, types, the build, the corpus and behaviour tests on sync, dist and web, publint, attw); the corpus cases are unchanged.Looked at and left alone
chunk()afterend()and a secondend()both succeed, and an event timestamp earlier than the previous event is accepted. These assume rules nobody has shown AWS enforces. The AWS pages for chunked uploads and event streams render client-side, so a fetch returned no text, and a search found no statement of either rule (only SDK source that chains each signature to the previous one). Left unchanged; reopen if someone shows a service rejecting such frames.end({})emits a trailer block whileend()does not: with...-PAYLOAD-TRAILERthe trailer block is what the service expects, so this reads as intended. Unchanged.nowpast year 9999 gives an invalidx-amz-date: unrealistic. Unchanged.docs/guide.mdandCHANGELOG.md(Unreleased) describe the new behavior.