Skip to content

feat(rpc): adopt oRPC v2 for service contracts#114

Closed
AmanVarshney01 wants to merge 1 commit into
mainfrom
aman/orpc-v2-rpc
Closed

feat(rpc): adopt oRPC v2 for service contracts#114
AmanVarshney01 wants to merge 1 commit into
mainfrom
aman/orpc-v2-rpc

Conversation

@AmanVarshney01

Copy link
Copy Markdown
Member

What this explores

This concept PR replaces Composer's hand-written RPC protocol layer with native oRPC v2 contracts, routers, and clients.

Composer still owns the differentiated parts: application topology, dependency hydration, per-edge service-key authorization, mount policy, and deployment. oRPC owns the generic RPC machinery: schema execution, transport, routing, structured errors, middleware, metadata, cancellation, and codecs.

Point-to-point example

Define one shared contract with normal oRPC syntax:

import { contract, oc } from '@prisma/composer/rpc'
import { type } from 'arktype'

export const calculatorContract = contract({
  double: oc
    .input(type({ value: 'number' }))
    .output(type({ result: 'number' })),
})

Implement it in the private provider:

import { implement, serve } from '@prisma/composer/rpc'

const rpc = implement(calculatorContract.router)

export default serve(calculatorService, {
  rpc: rpc.router({
    double: rpc.double.handler(({ input }) => ({
      result: input.value * 2,
    })),
  }),
})

Declare the service-to-service dependency:

export default compute({
  name: 'app',
  deps: {
    calculator: rpc(calculatorContract),
  },
  build: node({
    module: import.meta.url,
    entry: '../dist/app/index.mjs',
  }),
})

Use the injected typed client from any Fetch-capable framework, including Hono:

const { calculator } = appService.load()

app.get('/double/:value', async (c) => {
  const { result } = await calculator.double({
    value: Number(c.req.param('value')),
  })

  return c.json({ result })
})

The complete call is deliberately boring:

GET /double/21
  -> Hono app
  -> calculator.double({ value: 21 })
  -> { result: 42 }

What oRPC brings

  • native contract-first clients and Standard Schema transforms
  • nested routers without flattening method names
  • structured RPC errors with stable codes
  • middleware, metadata, cancellation, and request codecs
  • Fetch-native server/client transport and configurable body limits
  • a clean path to opt-in oRPC OpenAPI tooling without Composer inventing another protocol

The PR also migrates the existing Composer examples and adds a small end-to-end Hono + private Compute service example under examples/hono-rpc.

oRPC v2 is currently beta, so the runtime packages are pinned to the exact same version and Composer's wrapper stays intentionally small.

@wmadden-electric

Copy link
Copy Markdown
Contributor

Thanks for this PR — it's a serious piece of work, and evaluating it properly
forced us to firm up design decisions that were vaguer than they should have
been. The decision is not now: we're keeping the hand-written RPC layer
for the moment. But we do want oRPC in the picture later, in a different
shape than this PR, and I want to explain the reasoning properly rather than
just close it.

What we checked, and what held up

We went through your changes carefully, expecting to find problems, and
mostly didn't:

  • The type checking still works. The thing we care most about is that
    TypeScript rejects a bad wiring between services: a provider can offer
    more methods or richer results than a consumer asks for, but a provider
    that requires an input field the consumer never sends must not compile.
    We ran your migrated type tests against the branch and all of those cases
    behave correctly with oRPC's inferred client types. All the runtime tests
    pass too.
  • Interoperability doesn't require the library. A requirement of ours:
    two services must be able to talk as long as they agree on the contract
    and the wire protocol — neither side gets to force the other to run a
    particular npm package. oRPC clears this bar: the protocol doc
    (https://orpc.dev/docs/advanced/rpc-protocol) is enough to build a
    compatible client or server from scratch. One note: your ADR says
    hand-authored requests are not a supported client contract — we'd want
    the opposite stance, since that independence is the point.
  • You drew the responsibilities in the right place. oRPC handling
    validation, serialization, and routing while our code handles which
    services are connected, the per-connection auth key, and which procedures
    are actually published — that's the correct split, and details like
    running auth before the body is parsed are done well.

So this isn't "the code has problems". It doesn't, beyond a broken example
build in CI.

Why we're saying not now

You saw a reinvented wheel, and that's a fair reading. Here's the deliberate
part that isn't visible from the code: service-to-service calls in this
framework are trivial on purpose — a consumer calls
auth.verify({ token }) and gets a typed result, and that's the whole
story. Our RPC layer serves that with about four concepts and ten exports in
~420 lines.

Making oRPC the RPC layer means every user authors contracts and servers
with oRPC's API — which comes with its whole world: middleware, contexts,
plugins, links, a few hundred exported names. Your ADR argues that hiding
oRPC behind a small wrapper throws away the main reason to adopt an RPC
ecosystem. This is where we disagree about what the layer is for. We didn't
adopt RPC to give users an RPC framework. We adopted it for one narrow job:
strictly-checked connections between services, and a typed client generated
from a strict contract — it exists to support the topology. These are
internal service-to-service edges, not your application's API, and the
features that make oRPC attractive as an application API layer are exactly
the ones an internal edge doesn't need.

Worth being explicit: none of this limits what you build your app with.
Composer never touches your application code — if you want native oRPC for
the API your app serves to the internet, use it, today, as-is. This decision
is only about what the framework's own inter-service wiring imposes on every
user.

Where this work fits later

The direction we've landed on — partly because this PR forced the question:

  • The small built-in contract stays the default, and it's what consumers
    declare dependencies with: depend on the minimum you need.
  • oRPC becomes an optional richer way to build a provider. A module author
    who wants typed errors, middleware, or streaming builds their service with
    full native oRPC.
  • The connecting piece: a service built with rich oRPC should still satisfy
    a consumer who only declared a plain minimal contract. We've verified the
    type-level half of this already works — oRPC's inferred client type is
    assignable to our plain function signatures, with the variance rules
    intact. The runtime half needs design work we haven't done yet (checking
    contract compatibility across the two styles, and picking the right
    client transport per connection).

That's a bigger change than this PR, and it would be unfair to ask you to
rebuild toward a design that doesn't exist yet. So we're parking adoption
until it does. When it does, a lot of this PR carries over almost unchanged
— the wrapper, the serve() port checks, the auth interceptor, the tests.

What we're taking now

Comparing the two implementations exposed three gaps in our current code
that don't need oRPC to fix, and we're stealing them from your PR:

  1. a request body size limit,
  2. not leaking handler exception messages to callers,
  3. removing a redundant second validation of responses.

Thanks again — this genuinely moved the design forward even though it isn't
landing as-is.

wmadden pushed a commit that referenced this pull request Jul 20, 2026
Paths follow #131 (@internal/service-rpc, src/exports entrypoint). The
accepted residual is rejustified against connection-contracts scope —
internal means inside the topology, cross-network edges are in scope, and
robustness is calibrated against named failure modes — so the slice names
PRO-217 as what it guards and names the cross-instance window it leaves
open. The three fixes promised when #114 was declined are absorbed here
because they touch the same files. ADR-0037 reserved for the protocol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
wmadden pushed a commit that referenced this pull request Jul 20, 2026
…n it

serve() now rejects a keyless request with a loud 400 naming the
Idempotency-Key header. A duplicate key arriving while the first call for
it is still executing single-flights onto that same execution, so the
handler runs exactly once; a duplicate arriving after a completed 2xx/4xx
answer replays it byte-identically for 60 seconds. A 5xx is never cached,
since it's exactly the outcome a retry exists to re-execute. Storage is
keyed by method first, then by idempotency key, so a lookup for one
method can never find another method's cached answer, even if a key is
reused across methods. The cache is LRU-bounded at 1000 entries so it
stays a small, fixed amount of memory per provider process regardless of
traffic.

Handlers gain an optional third argument, ctx: RpcHandlerContext, carrying
ctx.idempotencyKey after the existing deps argument — every existing
two-argument handler still typechecks and runs unchanged. This is a
breaking protocol change: a caller that doesn't go through makeClient (the
client commit landing just before this one) must now supply its own key or
its calls 400.

This is permanent protocol semantics for this kind, not a compensation
tied to any one platform incident — see PRO-217 for the motivating urgency.
In-memory dedupe cannot cover a retry landing on a different instance than
the one that applied the call; that residual is accepted, and the
escalation path is a handler using ctx.idempotencyKey in its own
transaction where its target's failure modes warrant it.

Also lands the two remaining #114 fixes that touch this same dispatch
path:

- Request bodies are capped at 1 MiB, enforced against bytes actually read
  off the stream rather than the caller-supplied content-length (absent or
  lying, the cap still holds), and answered with 413.
- A handler throw or an output-validation failure now masks its message
  behind a generic 500 and logs the real error server-side, so an internal
  exception string never reaches the caller. Input-validation 400s are
  unchanged — that detail is the caller's own malformed request and is how
  they fix it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
wmadden-electric added a commit that referenced this pull request Jul 21, 2026
…#140)

* chore(drive): spec + dispatch plan for the rpc-cold-start slice

Contract-declared idempotence for rpc methods, the bounded client-side
retry for marked methods only, and the RPC face of the PRO-217 canary,
inheriting every rule from the cold-start canary's rebuild. Execution
held pending operator sign-off.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* chore(drive): the rpc slice becomes idempotency keys, not a boolean

Will's design supersedes the per-method idempotent flag: the protocol
requires a client-minted key on every call, serve() implements single-
flight and replay within the retry envelope, handlers may use the key for
durable control but need not. Every method becomes safely retryable with
no declaration to trust. Spec and plan rewritten; scope calibration and
the accepted cross-instance residual recorded in design-notes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* chore(drive): rebase the rpc slice onto the service-rpc rename

Paths follow #131 (@internal/service-rpc, src/exports entrypoint). The
accepted residual is rejustified against connection-contracts scope —
internal means inside the topology, cross-network edges are in scope, and
robustness is calibrated against named failure modes — so the slice names
PRO-217 as what it guards and names the cross-instance window it leaves
open. The three fixes promised when #114 was declined are absorbed here
because they touch the same files. ADR-0037 reserved for the protocol.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* chore: drop a stray file left at the pre-rename rpc path

An interrupted dispatch wrote protocol.ts under the old rpc/ directory;
it was untracked, so the rebase reset did not clear it and the previous
commit swept it in. The slice writes into service-rpc/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* feat(service-rpc): client mints an idempotency key and retries safely

makeClient() now mints one crypto.randomUUID() per logical call and sends
it as the Idempotency-Key header on every attempt, byte-identical across
retries of that call and never shared with another call. Every method
gets a bounded, jittered retry (250 ms initial, x2, 5 s cap, up to 5
retries): thrown network errors, 429, and any 5xx retry; any other 4xx
does not, since a malformed or unauthorized request stays wrong on a
second try. This exists because PRO-217 (the Prisma Compute ingress
closing a first-touch connection while a scale-to-zero target boots) hits
every service-to-service RPC edge raw today; a blanket retry was unsafe
until every call carried a key a server could dedupe on.

Also drops the client's redundant re-validation of the server's response
against the output schema: serve() already validates a handler's output
before responding, and both ends of every edge are framework-provisioned,
so the second pass only duplicated work. The MethodSchemas blindCast this
required falls out too — the client now only needs method names off
contract.__cmp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* feat(service-rpc)!: server requires the idempotency key and dedupes on it

serve() now rejects a keyless request with a loud 400 naming the
Idempotency-Key header. A duplicate key arriving while the first call for
it is still executing single-flights onto that same execution, so the
handler runs exactly once; a duplicate arriving after a completed 2xx/4xx
answer replays it byte-identically for 60 seconds. A 5xx is never cached,
since it's exactly the outcome a retry exists to re-execute. Storage is
keyed by method first, then by idempotency key, so a lookup for one
method can never find another method's cached answer, even if a key is
reused across methods. The cache is LRU-bounded at 1000 entries so it
stays a small, fixed amount of memory per provider process regardless of
traffic.

Handlers gain an optional third argument, ctx: RpcHandlerContext, carrying
ctx.idempotencyKey after the existing deps argument — every existing
two-argument handler still typechecks and runs unchanged. This is a
breaking protocol change: a caller that doesn't go through makeClient (the
client commit landing just before this one) must now supply its own key or
its calls 400.

This is permanent protocol semantics for this kind, not a compensation
tied to any one platform incident — see PRO-217 for the motivating urgency.
In-memory dedupe cannot cover a retry landing on a different instance than
the one that applied the call; that residual is accepted, and the
escalation path is a handler using ctx.idempotencyKey in its own
transaction where its target's failure modes warrant it.

Also lands the two remaining #114 fixes that touch this same dispatch
path:

- Request bodies are capped at 1 MiB, enforced against bytes actually read
  off the stream rather than the caller-supplied content-length (absent or
  lying, the cap still holds), and answered with 413.
- A handler throw or an output-validation failure now masks its message
  behind a generic 500 and logs the real error server-side, so an internal
  exception string never reaches the caller. Input-validation 400s are
  unchanged — that detail is the caller's own malformed request and is how
  they fix it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* fix(cron): give serve-schedule's test requests an idempotency key

serve() now requires every request to carry an Idempotency-Key header;
serveSchedule()'s production callers already get one for free since they
go through makeClient (the trigger dep hydrates via rpc()), but this
suite builds its trigger requests by hand and was failing 400 before
reaching the behavior each test actually exercises.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* docs(adr-0037): service RPC calls carry an idempotency key

Records the decision the slice implements: one key per logical call,
reused across retries; the server requires it, runs one call per key,
and replays completed answers but never 5xx. No per-method opt-in,
because a flag would be a claim the framework cannot verify. States
what the in-process store does not cover and why that residue is
deliberate, calibrated against connection-contracts scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* docs(connection-contracts): the network binding retries safely on a key

Adds the idempotency-key protocol to the network-binding section as a
property of the binding, not the contract, alongside the service key.
Corrects the per-call check: the server validates both directions; the
client no longer re-validates the response, since both ends are
framework-generated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* chore(drive): the rpc cold-start canary (PRO-217), plus a known limitation

scripts/rpc-cold-start-canary.ts + rpc-cold-start-canary-classify.ts (+
its test) mirror cold-start-canary.ts wholesale: create -> upload ->
start -> race the promote call, >=60s spacing including sample #0,
14-confirmed-cold-start-hold bug-gone budget, first-close early exit,
MAX_RUN_MS self-stop, requirable exits. The probe is a bare
single-attempt fetch straight at auth.service's POST /rpc/verify with
a manually minted Idempotency-Key, never through makeClient — this
slice just gave every framework RPC edge an automatic retry, and a
probe through that client would mask the raw platform behavior being
tested. New rpc-cold-start-canary job in e2e-deploy.yml, modeled on
cold-start-canary, same deploy-verify-destroy action and needs:
serialization, not added to any required-checks list.

auth.service never logged anything on boot, so there was no listening
line to read; examples/storefront-auth/modules/auth/src/server.ts now
logs one self-timestamped line right after Bun.serve() returns
(Compute's log relay passes plain app stdout through with no
platform-added timestamp, verified live, so the app has to stamp its
own, the same role streams-server's console.log patch plays for the
streams face).

auth.service is wired to exactly one RPC consumer (storefront), so per
ADR-0030/0031 it enforces a per-edge bearer key this script cannot
obtain: the Management API's own contract says an environment
variable's value "is stored encrypted and is not returned by
subsequent reads," confirmed live (an unauthenticated touch against a
warm instance got back exactly serve()'s documented 401). Sending no
Authorization header is the honest choice, and the classifier treats
that specific, known 401 the same as a 2xx for classification purposes,
since PRO-217's close happens at the ingress before serve()'s
accepted-key check ever runs.

Known limitation, evidenced by a 14-sample live run: auth.service's
boot (spark starting bun -> the app's own listening line) is
consistently well under 2s, so no touch's lead time can cross the
2000ms cross-clock-skew margin cold-start-canary-classify.ts's
technique requires before calling a touch confirmed-cold. Every touch
still raced a real boot (positive, consistent 376-1185ms gaps, 14/14
samples) and the close-decisive path is completely unaffected (a close
needs no boot evidence), so the canary can still catch bug-present, but
it cannot currently earn a bug-gone verdict against this target. Full
numbers and a recommendation are in this dispatch's report to the
orchestrator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* Revert "the rpc cold-start canary (PRO-217)" — the canary is not warranted

An RPC cold-start canary was dispatched but, on the evidence, dropped:

1. It cannot certify a cold start against examples/storefront-auth. The
   auth service restores no state, so it boots in under ~1.5s — narrower
   than the 2s clock-skew margin the coldness proof needs. The live run
   forced 14 real races (touch arrived 376-1185ms before listening, every
   time) yet certified none, correctly refusing to shrink the margin to fit.

2. More fundamentally, a cold-start canary exists to signal when a platform
   workaround can be deleted. The RPC idempotency retry is permanent
   protocol semantics (ADR-0037), not a workaround with an expiry — there
   is nothing for it to signal the removal of. And PRO-217 is a platform
   ingress bug, already watched by the streams cold-start canary, which
   detects it well precisely because streams has the long boot window RPC
   lacks.

The slice ships the keyed protocol; the streams canary remains the
platform-wide PRO-217 signal.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* chore(drive): record that the rpc canary was dropped on the evidence

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* docs(gotchas): PRO-217 gains the service-RPC face — keyed retry, no canary

The RPC client absorbs the cold-start close with a keyed, dedup-safe
retry that is permanent protocol semantics, so there is no workaround to
retire and no RPC canary; PRO-217 stays watched by the streams canary,
whose long boot window RPC lacks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* docs(guides): document the idempotency-key protocol on the RPC surface

building-an-app gains a "Calls retry safely for you" section (the key,
the safe retry, the keyless-400, the optional ctx third arg);
getting-started and SKILL get terse pointers to it. Matches the new
serve()/makeClient behavior from ADR-0037.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* fix(service-rpc): close the review findings on the keyed protocol

- A request body that errors mid-read now masks and logs like every other
  internal failure, instead of rejecting out of serve() (which broke its
  Response contract and skipped the operator log). New test, mutation-checked.
- Add the missing test for output-validation masking (a handler returning
  schema-violating output → masked 500 + provider-bug log), mutation-checked.
- Document why six retry attempts cover a boot up to ~22s (a held connection
  blocks until the server is listening; the budget only governs actively
  closed attempts) and that maxDelayMs is an unreached ceiling at maxRetries:5.
- standard-schema.ts no longer claims the client re-validates responses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* refactor(service-rpc): a keyless request opts out of dedup, not a 400

A request without an Idempotency-Key is now served once, without dedup or
replay, instead of being rejected. The safe path is the generated client,
which always pairs a key with the retry, so a keyless request never came
from it — accepting it adds no unsafe retry path and lets curl and older
callers work transparently, degrading gracefully instead of hard-breaking.
ctx.idempotencyKey becomes string | undefined.

Also: trim the oversized code comments down to what earns its place, and
rewrite ADR-0037 to build the accepted mechanism directly (the rejected
flag moves wholly to Alternatives) and to drop project-temporal framing.
The keyless-400 wording is updated across connection-contracts, the guides,
the skill, the ADR index, and the slice spec/plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* docs(service-rpc): trim the comments flagged in review

Remove the idempotency paragraph from serve()'s module header (the
IdempotencyStore doc and the dispatch code already carry it) and tighten
client.ts's header to its essentials.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

* fix(service-rpc): the LRU key separator was a literal NUL byte

serve.ts embedded a raw NUL (0x00) as the method/key separator in the
idempotency store's LRU key. A NUL makes git classify the whole file as
binary, so GitHub rendered "Binary file not shown" and hid the entire
diff. Written as the `\0` escape instead: identical runtime separator
(still collision-proof — neither a method name nor a UUID contains NUL),
text source.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>

---------

Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
@wmadden-electric

Copy link
Copy Markdown
Contributor

Closing per the evaluation above — not landing as-is, with the runtime-adoption door explicitly left open (the recorded future shape: oRPC as an optional richer provider surface beside the minimal default contract). The three gap-fixes this PR exposed in our hand-written layer are landing via #140. Thanks again — this PR moved the design forward materially.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants