Skip to content

feat(wire): split the frame discriminant into trait and method bytes - #357

Open
decrypto21 wants to merge 16 commits into
mainfrom
feat/wire-trait-method-split
Open

feat(wire): split the frame discriminant into trait and method bytes#357
decrypto21 wants to merge 16 commits into
mainfrom
feat/wire-trait-method-split

Conversation

@decrypto21

@decrypto21 decrypto21 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Merge gate (tracked in #351): all hosts on the Rust implementation. This is ready for review and current against main, not ready to merge ahead of that gate.

What

  • The frame discriminant becomes two u8s instead of one: the API trait (namespace) and the method within that trait.
    • before: [SCALE str requestId][u8 id][payload]
    • after: [SCALE str requestId][u8 trait][u8 method][payload]
  • Trait ids are explicit literals via a new trait-level #[wire_trait(id = N)] attribute, carried through the macro, rustdoc extraction, and both emitters. Missing or duplicate trait ids are hard codegen errors; there is no fallback that derives an id from declaration or module order.
  • Method ids restart from 0 within each trait, preserving each trait's existing relative order. The existing derivation rules are unchanged, now scoped per trait: a request takes n/n+1, a subscription takes n..n+3. Append-only becomes a per-trait rule.
  • Codec version goes 1 to 2 (--codec-version default, scripts/codegen.sh, the generated client constant, and the handshake's acceptance check).
  • Unknown discriminant pairs and undecodable frames now fail visibly instead of being dropped silently.
  • Trait ids are a dense range, system = 1 through locale = 16. No floor is reserved against codec 1's old flat numbering: codec 1 already assigned overlapping low ids of its own (e.g. Locale's flat ids 194-197), so a floor could not deliver the isolation it seemed to promise. A codec-1 peer is identified by its handshake codec version, not by trait-id range.

Why

  • The flat u8 is the whole address today, so new methods can only append at the global tail. Five traits already have non-contiguous id blocks as a result (account spans 18-28, 110-112, and 164), and ids 70-75 are permanently burned by the retired JsonRpc trait. 163 of 255 values are already assigned, and a single subscription costs four slots.
  • Splitting gives every trait its own 256-slot method space, keeps each trait's block contiguous permanently, and puts the namespace on the wire where dispatch and debug tooling can see it.
  • Since this is a coordinated cutover, existing ids are renumbered once and append-only resumes per trait. Preserving the old global numbers would carry today's fragmentation into the new scheme for no compatibility benefit: the envelope grows a byte, so no existing peer can parse a new frame regardless of which numbers it carries.

Compatibility

  • A peer on codec 1 talking to a new host gets an explicit UnsupportedProtocolVersion rather than a hang.
  • The reverse direction cannot be rescued by the version bump: the handshake frame itself rides the changed envelope, so an old peer cannot parse a new host's reply well enough to learn it is incompatible. It fails on its own side.
  • The codec bump plus the loud-failure changes exist so mismatches surface as errors instead of the silent-drop-then-hang failure this protocol produced the last time a byte layout moved under an unchanged version tag.
  • Phone pairing and the inter-host SSO messages are untouched: that layer carries a string message_id with its own codec indices and has no reference to this envelope's discriminant.

Nested envelope (RFC 0028)

Folds wire direction (request/response, or a subscription's start/stop/interrupt/receive) and version into the payload itself, so a method costs one wire id instead of two or four: truapi::versioned::Request<Req, Res> and Subscription<Start, Item, Err>, replacing RequestFrameIds/SubscriptionFrameIds above. Same unreleased codec 2 cutover, no third wire-breaking bump before release.

Every subscription's Interrupt carries a real, decodable Option<CallError<...>>, including plain (non-ResultSubscription) methods: the error type is derived structurally from the method's own envelope rather than defaulting to (). A framework-level decline of a host-initiated stream (e.g. an app declining a render) is encoded as Interrupt(Some(CallError::unavailable())), so a conforming peer sees it as a failure rather than a clean completion. A method whose request/item wrapper doesn't resolve to a {Base}Request/{Base}Item-shaped wire envelope is a hard codegen error in both emitters; neither has a directionless fallback payload to emit instead.

Verification

  • cargo fmt --all -- --check, cargo clippy --workspace --all-targets --all-features -- -D warnings, and cargo test --workspace --all-features are all clean: 1211 passed, 0 failed.
  • js/packages/truapi: tsc build clean, bun test 171 passed, 0 failed.
  • The truapi-codegen golden snapshot test (wire_table.rs, dispatcher.rs) passes against freshly regenerated output, nothing hand-edited, everything reproduced via ./scripts/codegen.sh and blessed from its own .actual dump.
  • The nested-envelope codegen path has no legacy fallback on either side: an envelope that can't be resolved is a build-time error in both dispatcher.rs and client.ts generation, not a silent directionless payload.
  • Byte-level goldens were recomputed rather than relaxed: the binary golden-account-get.bin fixture, the handshake and account_get_account wire-equality pins (Rust and TypeScript), and the Rust/TypeScript wire-table parity test all reflect the current (trait, method, version, direction) shape.
  • CI (this run) additionally covers: wasm32-unknown-unknown build of truapi-server, iOS bindings + Swift compile, Playground build/lint/unit, Explorer build/lint; not re-run locally in this pass, left to CI as this repo's own division of verification labor.

@decrypto21 decrypto21 closed this Aug 24, 2026
@decrypto21 decrypto21 reopened this Aug 28, 2026
@decrypto21
decrypto21 force-pushed the feat/wire-trait-method-split branch 3 times, most recently from 666a4ad to 78d84eb Compare August 28, 2026 08:03
@decrypto21
decrypto21 marked this pull request as ready for review August 28, 2026 08:11
@decrypto21
decrypto21 requested a review from a team August 28, 2026 08:11
@decrypto21
decrypto21 force-pushed the feat/wire-trait-method-split branch from 1b83f59 to 9bed8ac Compare August 31, 2026 08:18
@johnthecat

johnthecat commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Great work, thank you!
I have one additional proposal before public release - let's add 1 additional nesting and rearrange versioning.
I can create proper RFC, but main idea is this - hide requesting and subscription calls inside one additional Enum and move api versioning on top, so enumeration of methods will be straight increment instead weird gaps in wire index (0 -> 4 -> 6 -> 8). Wire format should Look like this:

enum TruApi {
    System(SystemPallet),
}

enum SystemPallet {
    Locale(LocaleMethod),
}

enum LocaleMethod {
    V1(Request<LocaleRequestPayloadV1, LocaleResponsePayloadV1>),
}

enum Request<Req, Res> {                                                                                                                                                                                         
    Request(Req),
    Response(Res),
} 

Same for subscriptions. As I said, I can create proper explanation in RFC.

What it gives us:

  • Simpler mental model for adding/removing/supporting methods.
  • Harder to reach enum values limit (256 values max in SCALE/JAM codecs)
  • Correct placement of version enum will help us to migrate between versions, now it's buried somewhere inside and will not help at all. If tomorrow we decide to move from plain request to subscription - it will be just version change inside method.

Why now, not as incremental update later:

  • Public release will effectively block us on such changes, after it all we can do is add/remove new methods, not change wire format structurally.

Comment thread rust/crates/truapi/src/api/account.rs Outdated

@pgherveou pgherveou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codec-2 review: four correctness issues remain around interrupt semantics and legacy codec-1 compatibility behavior.

Comment thread rust/crates/truapi-codegen/src/ts.rs Outdated
Comment thread rust/crates/truapi-codegen/src/ts.rs Outdated
Comment thread rust/crates/truapi/src/lib.rs Outdated
Comment thread rust/crates/truapi-codegen/src/ts.rs Outdated
main independently extended codec 1's flat numbering to 192 via
System::host_info, matching this branch's MIN_TRAIT_ID exactly and
breaking the invariant that a codec 1 frame can never look like a valid
codec 2 trait id. Every trait id shifts +1 (193..207), MAX_CODEC_1_METHOD_ID
moves to 192 and MIN_TRAIT_ID to 193 so the floor is strictly above the
known ceiling again, and every fixture, golden file, and generated
artifact that pinned the old numbers is regenerated or hand-updated to
match.
@decrypto21
decrypto21 force-pushed the feat/wire-trait-method-split branch from 94d95ff to 6ebb401 Compare September 2, 2026 09:28
@decrypto21 decrypto21 mentioned this pull request Sep 2, 2026
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants