Repository navigation
feat(wasm): streaming dispatch in the component adapter (issue #203) - #426
Merged
Merged
Conversation
…s#203) The cwist-guest-stream world had no JS entry point: the stream smoke called guestStream.dispatchStream directly and collected chunks in a hand-written host import. Consumers of cwist-wasm/component had nothing to move SSE code onto from the Emscripten Module.cwistStreamChunk path. createStreamHost() returns the c4punks:cwist/host import, and handle.stream(init, onChunk) routes each send-chunk to the callback of the stream in flight. The guest awaits each callback (backpressure). A throwing callback reaches the guest as the err arm of send-chunk, which aborts the pump, and stream() rejects with the callback's own error. Streams on one handle run one at a time and wait for a pending useSession(). component.d.ts now says handle() returns a Promise against 0.3 guests, and types the streaming additions. The stream smoke drives the guest through the adapter and adds the abort path. Refs c4punks#203
Member
|
Thanks for your contribution! |
- component.js kept STREAM_RUNNERS in module scope, but a bundler
consumer (and the stream smoke's esbuild alias) inlines a second copy
of the adapter into the guest bundle: the host registers with the
bundled copy while stream() looks up the npm copy, so streaming always
failed with 'streaming needs streamHost'. Keep the registry on
globalThis so every copy shares it.
- preview3-shim's preview2-shim io worker spawns
new Worker(new URL('./worker-thread.bundle.js', import.meta.url));
esbuild does not emit that file, so component-stream-smoke copies it
next to the bundle.
Member
|
Also we have discord here :) |
Member
|
now it's merged |
1 of 2 tasks
gg582
added a commit
that referenced
this pull request
Oct 22, 2026
…427) to main (#434) * feat(wasm): streaming dispatch in the component adapter (issue #203) (#426) * feat(wasm): streaming dispatch in the component adapter (issue #203) The cwist-guest-stream world had no JS entry point: the stream smoke called guestStream.dispatchStream directly and collected chunks in a hand-written host import. Consumers of cwist-wasm/component had nothing to move SSE code onto from the Emscripten Module.cwistStreamChunk path. createStreamHost() returns the c4punks:cwist/host import, and handle.stream(init, onChunk) routes each send-chunk to the callback of the stream in flight. The guest awaits each callback (backpressure). A throwing callback reaches the guest as the err arm of send-chunk, which aborts the pump, and stream() rejects with the callback's own error. Streams on one handle run one at a time and wait for a pending useSession(). component.d.ts now says handle() returns a Promise against 0.3 guests, and types the streaming additions. The stream smoke drives the guest through the adapter and adds the abort path. Refs #203 * fix(wasm): share the stream runner registry and ship the io worker - component.js kept STREAM_RUNNERS in module scope, but a bundler consumer (and the stream smoke's esbuild alias) inlines a second copy of the adapter into the guest bundle: the host registers with the bundled copy while stream() looks up the npm copy, so streaming always failed with 'streaming needs streamHost'. Keep the registry on globalThis so every copy shares it. - preview3-shim's preview2-shim io worker spawns new Worker(new URL('./worker-thread.bundle.js', import.meta.url)); esbuild does not emit that file, so component-stream-smoke copies it next to the bundle. --------- Co-authored-by: Lee Yunjin <gzblues61@daum.net> * feat(build): CWIST_FAT/CWIST_APP_FAT embedded-friendly fat archive (#427) * feat(build): CWIST_FAT/CWIST_APP_FAT embedded-friendly fat archive - make CWIST_FAT=1 folds every vendored static archive (BoringSSL, lsquic, libttak, cnats, cJSON, uriparser, usrsctp) into libcwist.a via an ar MRI merge (libtool -static on Darwin), so copying the single archive plus include/cwist onto an embedded target is enough to link a server app; only system libraries stay external. A .build-fat stamp re-archives when the toggle flips, without recompiling objects. - CWIST_APP_FAT=1 implies CWIST_FAT=1 and drops the vendored archives from the example apps' link line (APP_LIBS): the app binary pulls everything from libcwist.a. - New fat.yml workflow builds the fat archive, checks vendored members and key symbols are folded in, links and serves the hello example in a scratch dir with only libcwist.a + headers + app source present, and builds an example with CWIST_APP_FAT=1. * fix(build): select the fat recipe at recipe time and stage fat-dist - The ifeq-selected recipe was chosen at parse time, so fat-dist's target-specific CWIST_FAT=1 built a plain archive with a fat stamp; the embedded link then missed BoringSSL (SSL_get_error via the system OpenSSL DSO). Branch on $(CWIST_FAT) inside the recipe instead. - New fat-dist target stages dist/cwist-fat/: the fat archive plus the vendored headers the public API references (cjson, openssl, sqlite3, ttak) — the bundle copied to the embedded board. - fat.yml now builds fat-dist and runs the link smoke from the bundle. --------- Co-authored-by: anux <anushree.kasturi06@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #203. This does not close the issue: the Emscripten cutover is still gated on unflagged JSPI, as recorded in
docs/api/wasm-component.md.What changed
The
cwist-guest-streamworld had no JS entry point. The stream smoke calledguestStream.dispatchStreamdirectly and collected chunks in a hand-written host import, socwist-wasm/componentusers had nothing to move SSE code onto from the EmscriptenModule.cwistStreamChunkpath.wasm/npm/component.js:createStreamHost()returns thec4punks:cwist/hostimport ({ sendChunk }).createCwistFromComponent()acceptsdispatchStreamandstreamHost, andhandle.stream(init, onChunk)routes each chunk to the callback of the stream in flight.errarm ofsend-chunk(thrown{ payload }, which jco's import wrapper turns intoerr). The guest aborts the pump, andstream()rejects with the callback's own error instead of the generic dispatch-error.useSession().handle()anduseSession()behave as before.wasm/npm/component.d.ts:handle()is typed as returningCwistResponse | Promise<CwistResponse>(it already returns a Promise against 0.3 guests), and the streaming additions are typed.tests/wasm_component_stream_host.js/tests/wasm_component_stream_test.js: the smoke now drives the guest through the adapter, with an awaiting callback, and adds the abort path (callback throws on the first chunk,stream()rejects with that error after exactly one chunk, and a followinghandle()still returns 200).wasm/npm/README.md,docs/api/wasm-component.md: document the API.How it was verified
useSession()beforestream(), backpressure, callback error vs guest error, two concurrent streams not interleaving, a failed stream not blocking the next, clear errors for a missingdispatchStream/streamHost/ callback, and the sync 0.2 path unchanged. All pass on Node v24.16.0.esbuild --bundle --platform=node --format=esmwith--alias:c4punks:cwist/host=tests/wasm_component_stream_host.js(same flags ascomponent-stream-smoke): the bundle loads andsendChunkis the adapter's host function.tsc --noEmit --strict wasm/npm/component.d.ts(TypeScript 5.6.3): clean.node --check tests/wasm_component_stream_test.js,git diff --check, added lines are ASCII.make component-stream-smoke-p3(no wasi-sdk 34 / wasm-tools / wit-bindgen on this machine). That job in the WASM workflow is the end-to-end check for the real guest, including the abort path.Not in this PR
index.js).