feat(service-rpc): idempotency keys make every RPC call safe to retry

.drive/projects/forcing-function-apps/design-notes.md

@@ -191,3 +191,57 @@ shipped inside Part B's refactor (implicit create, read-creates, contentType
 dropped), the complete unenforced-invariant inventory (T1–T9), deviations
 from the design doc, and decisions R1–R6 awaiting his ruling. No further
 dispatches until he rules.
+
+## RPC idempotency keys — settled design (2026-07-17, Will)
+
+Supersedes the `idempotent: true` per-method boolean considered earlier the
+same day. The RPC protocol requires an `Idempotency-Key` on every call
+(client-minted per logical call, reused across retries); `serve()` enforces
+it, runs single-flight per key, and replays completed answers within the
+retry envelope; handlers may read `ctx.idempotencyKey` for their own durable
+control but need not. Rationale: a boolean is a human claim the framework
+cannot check — the key is a mechanism it enforces, making every method
+safely retryable with no declaration. Scope calibration from Will: this
+protocol serves same-network service-to-service calls and guards the
+cold-start bug specifically; in-memory control is sufficient, and the
+cross-instance residual window is documented and accepted, not closed.
+The bounded retry becomes permanent protocol semantics — the PRO-217 canary
+retires itself when the platform heals, never the retry or the keys.
+Slice: [slices/rpc-cold-start/](slices/rpc-cold-start/spec.md).
+
+### Amendment (2026-07-17, later): rename, scope citation, absorbed fixes
+
+The kind was renamed to service-rpc (#131, ADR-0036) while this slice was in
+flight; the branch is rebased and all paths follow the new package and the
+`src/exports/` entrypoint convention (ADR-0035). Two corrections from Will:
+
+- The accepted residual is no longer justified by "same network" — that
+  framing contradicts `connection-contracts.md` § Purpose and scope, which
+  defines internal as membership of the application's topology and puts
+  cross-target, cross-network edges explicitly in scope, with robustness
+  calibrated per edge against the named failure modes of the targets
+  carrying it. The slice now names its concrete failure (PRO-217's close
+  during connection establishment, before any handler runs) and names the
+  residual it does not close (a retry reaching a different instance than the
+  one that applied the call), whose escalation path is the handler's own
+  durable control via `ctx.idempotencyKey`.
+- Three fixes promised when #114 was declined touch the same files this
+  slice rewrites, so they are absorbed rather than left to collide: a
+  request body size limit, masking handler exception messages instead of
+  returning them to callers, and dropping the client's redundant second
+  validation of responses.
+
+ADR-0037 is the number for the keyed protocol (0033–0036 taken); it ships
+with the implementation in this slice's PR, never as a docs-only change.
+
+### RPC cold-start canary dropped (2026-07-20, Will accepted)
+
+The slice's own canary was built, run live, and dropped on the evidence
+(revert `a46abfa`). The auth service boots in under ~1.5s (no state to
+restore), narrower than the 2s coldness-proof margin, so 14 forced races
+certified as cold zero times. The decisive reason is not the margin, though:
+the RPC idempotency retry is permanent protocol semantics (ADR-0037), so
+there is no workaround for a canary to time the removal of — and PRO-217, a
+platform ingress bug, is already watched by the streams canary, which catches
+it well because streams has the long boot window RPC lacks. The slice ships
+the keyed protocol; the streams canary stays the platform-wide PRO-217 signal.

.drive/projects/forcing-function-apps/slices/rpc-cold-start/plan.md

@@ -0,0 +1,94 @@
+# Dispatch plan: rpc-cold-start (idempotency keys)
+
+Contract source: [spec.md](spec.md). Branch:
+`claude/streams-cold-start-rpc-37e5c1`, rebased onto main at/after `6ec2625`
+(the [#131](https://github.com/prisma/compose/pull/131) service-rpc rename).
+Three dispatches, sequential; hostile reviewer round after D1+D2, then D3
+closes. Orchestrator owns all docs (`gotchas.md`, `docs/`, `.drive/`, and
+ADR-0037) — implementers report staleness and never edit. Evidence rules from
+the streams slice apply verbatim: raw program output only; every reported
+number is checked against the code's real format strings before it is
+believed.
+
+## D1 — the keyed protocol, plus the three absorbed #114 fixes
+
+**Outcome:** in `packages/0-framework/2-authoring/service-rpc/`:
+`makeClient` mints one `Idempotency-Key` per logical call and reuses it
+across a bounded retry (250 ms / ×2 / 5 s cap / 5 attempts / jitter; retry
+network errors + 5xx + 429, never other 4xx). `serve()` enforces the key
+(keyless requests opt out — served once, no dedup), single-flights per in-flight key, replays completed
+2xx/4xx for ~60 s under a justified LRU bound, never caches 5xx/throws, and
+passes `ctx.idempotencyKey` as an optional third handler argument after
+`deps`. Absorbed in the same pass: the request body size limit (413,
+enforced while reading, not trusting `content-length`), masking handler and
+output-validation exception messages behind a generic 500 while logging the
+real error, and deleting the client's redundant response re-validation
+(with the now-unnecessary `MethodSchemas` cast if it falls out). No
+prisma-cloud import; no `idempotent` flag anywhere.
+
+**Completed when:** every client, server, and type-level test in the spec is
+green with teeth confirmed red-by-mutation (including same-key-across-
+attempts, fresh-key-per-call, 5xx-not-cached, and exception-message-not-
+returned); both RPC-consuming example suites pass unchanged; repo checks
+green with casts delta ≤ 0; committed with DCO dual sign-off.
+
+## D2 — the canary (scripts + CI) — ATTEMPTED, THEN DROPPED (2026-07-20)
+
+**Outcome: no RPC canary ships.** The canary was built and run live; the
+evidence retired it (reverted in `a46abfa`). Two reasons, the second
+decisive:
+
+1. It cannot certify a cold start against `examples/storefront-auth`. The
+   auth service restores no state and boots in under ~1.5s — narrower than
+   the 2s clock-skew margin the coldness proof requires. The live run forced
+   14 genuine races (touch landed 376–1185ms before the listening line,
+   every sample) but certified none, correctly refusing to shrink the margin
+   to make its own numbers pass.
+2. A cold-start canary exists to signal when a platform workaround can be
+   deleted. The RPC idempotency retry is **permanent protocol semantics**
+   (ADR-0037), not a workaround — there is nothing for a canary to time the
+   removal of. PRO-217 itself is a platform ingress bug already watched by
+   the streams cold-start canary, which catches it well precisely because
+   streams has the long boot window RPC lacks.
+
+Superseded original intent (kept for the record): a sibling of the streams
+cold-start canary probing the auth service's rpc endpoint with a bare
+single-attempt `fetch`. If a future RPC edge with a real restore-from-store
+boot exists, and only if the retry ever becomes removable, revisit with an
+independently measured skew margin rather than the streams canary's borrowed
+constant.
+
+### Original D2 outcome (not shipped)
+
+**Outcome:** `scripts/rpc-cold-start-canary.ts` + `-classify.ts` + unit
+tests, inheriting the cold-start canary's proven contract wholesale
+(promote-race trigger, ≥60 s spacing including sample #0, log-confirmed
+coldness with the 2 s margin, 14-hold bug-gone budget, first-close early
+exit, `MAX_RUN_MS`, requirable exits, and a bug-gone message that retires
+the canary and its gotchas paragraph but NEVER the retry or keys); job in
+`e2e-deploy.yml` over `examples/storefront-auth`, probing the auth service's
+rpc endpoint with a bare single-attempt `fetch` and a manually minted key —
+never through a framework client, which the new retry would mask.
+
+**Completed when:** classify tests green with confirmed teeth;
+`test:scripts` green; at least one live run reports `bug-present` with raw
+per-sample output (a clean run today means a broken canary — stop and
+report, do not ship); workspace left clean with project counts; committed.
+
+## D3 — hostile review, live re-proof, records, PR
+
+**Outcome:** reviewer pass over D1+D2. Attack priorities: a repeated key can
+never double-execute within an instance; a replay can never leak one logical
+call's answer to another, across methods or callers; a keyless request opts
+out cleanly (served once, no dedup); the body cap holds without a truthful `content-length`;
+no handler exception text reaches a caller while operators keep it; the
+canary cannot be masked by the retry machinery; every reported number is
+real. Findings closed. Full live round (deploy storefront-auth, canary
+verify, destroy, zero leaks).
+
+Then the orchestrator writes, in this same PR: **ADR-0037** (calls carry an
+idempotency key; the framework dedupes; retries are protocol semantics, with
+the named residual and the connection-contracts scope calibration as its
+reasoning), the `connection-contracts.md` protocol update, and the
+`gotchas.md` PRO-217 service-RPC face. PR opened against main with the slice
+narrative; review requested from Will. No auto-merge; merge on his word.