spike(L2): thin per-session gateway over OpenRouter — Contract #1 [DO NOT MERGE] - #486
Draft
ZIJ wants to merge 1 commit into
Draft
spike(L2): thin per-session gateway over OpenRouter — Contract #1 [DO NOT MERGE]#486ZIJ wants to merge 1 commit into
ZIJ wants to merge 1 commit into
Conversation
Slice-0 de-risk spike for the Flue-native agent type (oc-bg-agents work/flue-native-buildout.md, design 013 §4). Proves a THIN CF Worker over OpenRouter and freezes inter-lane Contract #1 (the gateway HTTP contract). EXTENDS the shipped managed-model path, does not replace it: org-level spend keeps flowing through the org's single OpenRouter inference key → the existing model_meter cron → Autumn (one cost-source-of-truth). The gateway only adds the injection point a CF Worker needs (can't use the box secrets-proxy) plus per-session sub-metering + on-path budget enforcement. Worker (cloudflare-workers/oc-gateway/): - index.ts — verify per-session token → (org,agent,session); budget gate on-path; inject the ORG's OR key (never exposed to the tenant); forward to openrouter.ai/api; tee-meter the response cost; passthrough (JSON + SSE). - token.ts — HS256 session token (Web Crypto). PROD: EdDSA + lease-epoch fence. - budget.ts — SessionBudget DO: strongly-consistent per-session µ$ counter + gate (a DO, not KV, so concurrent calls can't double-spend past the cap). - cost.ts — per-response cost from OR's echoed usage.cost (JSON + SSE). - scripts/mint.ts — mint a session token for live verification. Explicit answer (per-session OR keys vs on-path counter): ON-PATH COUNTER. Per-session OR keys would fragment the per-org cost-source-of-truth the cron reconciles + add per-session key lifecycle; the gateway is already on-path so it reads OR's echoed cost and enforces before/after each call (authoritative, not best-effort observe()). Cost = a bounded one-call overshoot, proven in tests. Verification: 14 vitest green — logic (token/cost) + integration (real handler + real DO vs mock OpenRouter): 401s, org-key injection, usage.include, on-path budget refusal with bounded overshoot, uncapped sessions. One REAL anthropic/* turn is documented (no OR key/dev env here; the buildout allows documenting it) — README has the exact wrangler-dev + registerProvider steps. opencomputer = PR-only; do NOT merge (Igor merges). Draft for review. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
suryatmodulus
pushed a commit
to suryatmodulus/opencomputer
that referenced
this pull request
Aug 2, 2026
Productionizes the 1a spike (spike/oc-gateway, diggerhq#486) into the flue-native gateway per buildout contract diggerhq#1 / design 013 §4. Thin OC Worker over OpenRouter: verify a per-session token → epoch-fence + on-path budget gate → inject the org's OR key → cache-safe + usage-accounted forward → sub-meter. Org spend stays on the org's single OR key → the existing OpenRouter→Autumn cron (nothing pushed to Autumn); the gateway only adds per-session sub-metering + enforcement. What changed vs the spike: - Token: HS256 → EdDSA (Ed25519). The minter holds the private key; the gateway holds only GATEWAY_TOKEN_PUBLIC_KEY (base64url raw). Alg pinned to EdDSA (rejects none/HS256 swaps). - Lease-epoch fence (close the freeze-flags): the SessionBudget DO tracks a monotonic max_epoch; a token with a stale `ep` → 401 token_superseded; a newer epoch supersedes older in-flight tokens. DO-serialized. - Org OR key from the credential store, not the KV/SPIKE stand-in: a dedicated internal sessions-api seam (mirrors the edge's dedicated-secret key hand-off), cached per-org in-isolate (60s). TEST_OR_KEY override for the acceptance run. The L3 route (resolveManagedSecret) is flagged for sessions-api. - Prompt-caching safety: strip cache_control for caching-unsafe models (claude-3-haiku → Bedrock 400s) via an env-extensible denylist. - Kept: usage:{include:true}, 402 budget_exceeded, metering by token.sub, the µ$ counter + /add idempotency on the OR generation id. Tests: 23 green (vitest) — EdDSA/alg-pin/tamper/expired, SessionBudget fence + budget + idempotency, cache_control strip, cost extraction, and the full on-path flow through the real handler. Live acceptance (2026-07-05, real OpenRouter via wrangler dev --local, throwaway $1-capped OR key minted from the provisioning key and deleted after): a real claude-haiku-4.5 turn completed gateway → OpenRouter (200, no key in the response, token != key); per-session budget refused on-path (402); cache_control stripped so claude-3-haiku succeeds via the gateway where a direct OR call 400s; epoch fence returns 401 token_superseded. Seam to confirm with the orchestrator (W1): EdDSA claim set + the mint side (default: per-turn token via getApiKey). L3 seam to build: the internal org-OR-key route. Not merged — Igor merges opencomputer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Slice-0 de-risk spike for the Flue-native agent type (
oc-bg-agents work/flue-native-buildout.md, design013 §4), Lane L2. Proves a thin CF Worker over OpenRouter and delivers frozen inter-lane Contract #1 (the gateway HTTP contract). Draft — opencomputer is PR-only, Igor merges.The full contract + live-verification steps are in
cloudflare-workers/oc-gateway/README.md.The proof
npx vitest run→ 14 green.test/integration.test.tsruns the real worker handler + realSessionBudgetDO against a mock OpenRouter and asserts the whole on-path flow: 401 (missing/expired token), forward with org-key injection (the session token never reaches OR) +usage.include, body passthrough, on-path budget enforcement with a bounded one-call overshoot ($0.03 cap, $0.02/call → 2 pass, 3rd → 402), uncapped sessions never refuse.One real
anthropic/*turn isn't run here (no OpenRouter key / dev env in this sandbox — the buildout explicitly allows documenting it); the README has the exactwrangler dev+registerProvider(baseUrl=gateway)steps. The OR wire format (cost echo) is already validated by the §9.7 managed-path spike.Extends, doesn't replace
Org-level spend keeps flowing through the org's single OpenRouter inference key → the existing
model_metercron → Autumn (one cost-source-of-truth). The gateway builds no billing path; it only adds the injection point a CF Worker needs (it can't use the box secrets-proxy) + per-session sub-metering + enforcement.Explicit answer: per-session OR keys vs on-path counter → on-path counter
Per-session OR keys would fragment the per-org truth the cron reconciles and add a per-session key lifecycle at session scale; the gateway is already on-path, so it reads OR's echoed cost and enforces before/after each call (authoritative, not best-effort
observe()). ASessionBudgetDO (not KV) gives read-then-write consistency so concurrent subagent/tool calls can't double-spend past the cap.Flagged for the frozen contract / prod
SPIKE_OR_KEY/ORG_KEYSKV stand-in → prod fetches the sealed OR key from the credential store (Infisical), cached per-org.402 budget_exceeded— the exact shape Flue/pi-ai surfaces cleanly as a turn-terminating error (→ outcomebudget_exceeded) is a live-verify item.usage.cost;GET /generation?id=exact-cost fallback documented, unwired.🤖 Generated with Claude Code