Skip to content

README.md

Latest commit

 

History

History
321 lines (273 loc) · 15.2 KB

README.md

File metadata and controls

321 lines (273 loc) · 15.2 KB
Raw

jeb

A TypeSafe Jev-compatible API on Cloudflare Workers, powered by Inception Mercury 2.5 through structured outputs, served either through OpenRouter or Inception's direct API.

Send a state and a map of typed questions; get back the same answers shapes Jev returns: choice with a probability distribution and confidence, score with a probability-weighted value and legend, and noul with a yes-probability. Existing TypeSafe client SDKs work against it by pointing the base URL at your deployment.

Full HTTP reference: docs/api.md.

How it works

  1. The request is validated against the TypeSafe wire shape; errors mirror TypeSafe's 401/422/429/529 status codes.
  2. One JSON Schema is built for the response: each question becomes a positional entry (q0, q1, ...) with a fixed-length probability array.
  3. All questions are evaluated against the same state in a single Mercury 2.5 call with response_format: { type: "json_schema", strict: true }.
  4. The model's probabilities are normalized, choice is the argmax, score is the probability-weighted level average, and confidence is computed from the distribution with the formula TypeSafe uses in its confidence explorer, (n * peak - 1) / (n - 1), clamped to [0, 1].
  5. Answers are re-keyed under the question ids you chose and returned with model and usage.

Question ids never reach the model. Every question is answered in the same request, which is Jev's one-request fan-out pattern. The trade-off is that the questions share one generation rather than being evaluated on independent passes, so this server is a shape-compatible stand-in rather than a behavioral replica.

Endpoints

Method Path Auth Description
POST /v1/systemone API_KEY Evaluate state against typed questions
GET /v1/models API_KEY List the configured model and compatibility aliases
GET /health none Liveness probe
GET / none Server info

API_KEY means the route requires Authorization: Bearer <API_KEY> when the server's API_KEY variable is set. GET /v1/models, GET /health, and GET / are not part of the TypeSafe API; the rest mirror it. See docs/api.md for request and response field tables.

Quick start

npm install
cp .dev.vars.example .dev.vars   # then set the key for your provider
npm run dev
curl -X POST http://127.0.0.1:8787/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": "Hi, I have been trying to connect my Stripe account for 3 days and the integration keeps failing. I am losing sales. Please help ASAP.",
    "model": "jev-latest",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this",
        "criteria": {
          "billing": "Payment or subscription issues",
          "technical": "Bugs or integration problems",
          "sales": "Pricing or account questions"
        }
      },
      "frustration": {
        "type": "score",
        "instructions": "How frustrated the customer appears",
        "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"]
      },
      "is_urgent": {
        "type": "noul",
        "instructions": "The message conveys urgency or time-sensitivity"
      }
    }
  }'
{
  "model": "inception/mercury-2.5",
  "answers": {
    "department": {
      "type": "choice",
      "choice": "technical",
      "probabilities": { "billing": 0.05, "technical": 0.9, "sales": 0.05 },
      "confidence": 0.85
    },
    "frustration": {
      "type": "score",
      "score": 1.05,
      "legend": {
        "0": "Calm, just stating facts",
        "1": "Frustrated but civil",
        "2": "Very angry, strong language"
      },
      "probabilities": { "0": 0.05, "1": 0.85, "2": 0.1 },
      "confidence": 0.775
    },
    "is_urgent": { "type": "noul", "noul": 0.95 }
  },
  "usage": { "input_tokens": 1172, "output_tokens": 138 }
}

The Authorization header is optional until you set API_KEY on the server. The SDKs always send one, so the same request works either way.

Drop-in client SDKs

The official SDKs work against this worker by overriding the base URL. They already send model: "jev-latest", which this server accepts and resolves to the configured model of the active provider.

JavaScript:

import { TypeSafeClient, choice, noul, score } from "@typesafe-ai/sdk";

const client = new TypeSafeClient({
  apiKey: process.env.TYPESAFE_API_KEY,
  baseURL: "http://127.0.0.1:8787", // or your deployed worker URL
});

const { answers } = await client.systemOne({
  state: "My running shoes arrived in the wrong size. Can I swap them for a size 10?",
  questions: {
    department: choice("Which team should handle this?", {
      returns: "Exchanges, wrong or damaged items",
      shipping: "Delivery status, delays, lost packages",
      billing: "Charges, invoices, payment problems",
    }),
    wants_exchange: noul("Does the customer ask to exchange the item?"),
    frustration: score("How frustrated is the customer?", ["Calm", "Frustrated", "Very angry"]),
  },
});

Python:

TYPESAFE_BASE_URL=http://127.0.0.1:8787 TYPESAFE_API_KEY=local python app.py

Both TYPESAFE_BASE_URL and TYPESAFE_API_KEY are the SDKs' own environment variables; the client sends the key as a bearer token and this worker checks it only when API_KEY is configured (see below). The JavaScript SDK flow above was verified against @typesafe-ai/sdk 0.6.0, including typed answers, usage, and 422-to-UnprocessableEntityError mapping.

Configuration

Variable Required Default Purpose
JEB_PROVIDER no openrouter Upstream for model calls: openrouter or inception (direct API)
OPENROUTER_API_KEY yes* – Key used for model calls when JEB_PROVIDER=openrouter
OPENROUTER_MODEL no inception/mercury-2.5 OpenRouter model slug
MODEL_ID no – Legacy alias for OPENROUTER_MODEL, honored when it is unset
INCEPTION_API_KEY yes* – Key used for model calls when JEB_PROVIDER=inception
INCEPTION_MODEL no mercury-2.5 Direct Inception API model id
INCEPTION_BASE_URL no https://api.inceptionlabs.ai/v1 Inception API base URL override
API_KEY no – (server is open) When set, clients must send Authorization: Bearer <API_KEY>
REASONING_EFFORT no reasoning disabled Server default effort: none, low, medium, or high
TEMPERATURE no 0 Sampling temperature
MAX_TOKENS no – Completion token cap

*Required for the selected provider; the other provider's key can stay unset.

Model providers

JEB_PROVIDER chooses where model calls go. Both routes send the same prompt and JSON Schema; they differ in the endpoint, the model id, and how reasoning is expressed:

  • openrouter (default) calls OpenRouter's chat-completions API with OPENROUTER_API_KEY and the OPENROUTER_MODEL slug (or legacy MODEL_ID), defaulting to inception/mercury-2.5. Reasoning goes out as OpenRouter's reasoning: { effort } field.
  • inception calls Inception's direct OpenAI-compatible API with INCEPTION_API_KEY and the INCEPTION_MODEL id, defaulting to mercury-2.5; override the endpoint with INCEPTION_BASE_URL. Reasoning goes out as an OpenAI-style reasoning_effort string, and none is always sent so answers stay fast by default.

Either way, reasoning stays off unless a caller asks for it (see Reasoning effort).

Local development reads these from .dev.vars. For production:

npx wrangler secret put OPENROUTER_API_KEY  # when JEB_PROVIDER=openrouter
npx wrangler secret put INCEPTION_API_KEY   # when JEB_PROVIDER=inception
npx wrangler secret put API_KEY             # optional
npx wrangler deploy

Reasoning effort

POST /v1/systemone accepts an optional reasoning_effort field (reasoningEffort is accepted as an alias). It overrides the server default for that request. Mercury 2.5's supported levels, per its OpenRouter model metadata, are:

Value Behavior
none Reasoning disabled
low Small reasoning budget
medium Moderate reasoning budget
high Large reasoning budget

Anything else (minimal, max, xhigh, ...) returns 422.

curl -X POST http://127.0.0.1:8787/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": "This is the third time I am writing in and honestly I am done.",
    "reasoning_effort": "high",
    "questions": { "angry": { "type": "noul", "instructions": "Is the customer angry?" } }
  }'

Measured end-to-end through this worker with one representative four-question request (choice + two scores + noul), Mercury 2.5 via OpenRouter, September 2026. Inception's shared pool rate-limited sustained runs partway through, so treat these as indicative rather than a rigorous benchmark:

Effort Latency Output tokens Reasoning tokens
none / unset 0.6–2.2 s (n=12, median ~0.9 s) 190–366 0 (measured)
low 0.8–1.4 s (n=6, median ~1.3 s) 398–478 251–258 (measured)
medium 2.5–3.1 s (n=2) 1,460–2,402 1,325 (measured on one run)
high 3.7 s (n=1) 1,553 1,427

Measured the same way through the direct Inception API (JEB_PROVIDER=inception), 21 September 2026, 22 serial requests with no rate limiting. Inception reports a single completion token count, so reasoning tokens are included in output and not itemized:

Effort Latency Output tokens (incl. reasoning)
none / unset 0.55–1.55 s (n=10, median ~1.1 s) 172–233
low 1.19–2.90 s (n=6, median ~2.3 s) 405–452
medium 2.07–8.09 s (n=4, median ~5.9 s) 1,178–1,561
high 2.18–6.33 s (n=2) 1,495–1,636

Rough rule of thumb: low costs a few hundred milliseconds over none, medium doubles to triples latency, and high adds a bit more on top of medium for a similar token count on this input. Through the direct Inception API, low roughly doubles the none latency instead, and medium/high run several times slower with much more run-to-run variance. Raise effort only for questions where a gut call keeps landing on a boundary; keep the default for everything else.

Compatibility notes

  • Probabilities are instructed, not natively calibrated. Jev is trained for calibrated decisions; Mercury 2.5 is prompted to distribute probability mass honestly. Shapes and math match Jev, but thresholds tuned against real Jev values may need retuning.
  • confidence is computed from the returned distribution with the formula TypeSafe's confidence explorer uses, (n * peak - 1) / (n - 1), clamped to [0, 1]. It is not rounded to two decimals, so it can show more precision than TypeSafe's example responses.
  • probabilities are normalized to sum to 1 and rounded to six decimals, with the rounding residue placed on the largest value.
  • usage.output_tokens includes reasoning tokens when reasoning is enabled, matching the provider's completion token count.
  • model reports the model that answered — for example inception/mercury-2.5 via OpenRouter or mercury-2.5 through the direct Inception API — rather than a jev-* version id.
  • The model field is advisory. Any accepted name resolves to the active provider's configured model (OPENROUTER_MODEL or INCEPTION_MODEL); aliases like jev-latest are listed by GET /v1/models for compatibility.
  • Validation enforces TypeSafe's documented limits: 2–10 Score levels and at most 255 Choice options. Request-size limits are not enforced.
  • Errors mirror TypeSafe's status codes: 401 missing/invalid key, 422 invalid body (with an error.param naming the field), 429 rate limited (with retry-after passed through), 529 overloaded. Upstream failures map to 429/529/502.
  • Reasoning is disabled by default for the default model so responses stay fast, like a System One model. Set reasoning_effort per request (or the REASONING_EFFORT server default) to let Mercury think first; see Reasoning effort for measured latency differences.

Project layout

docs/
  api.md         Full HTTP reference: fields, answers, errors, deviations
src/
  index.ts       Worker entry: routing, auth, CORS, error responses
  provider.ts    Model provider call (OpenRouter or direct Inception API),
                 structured-output wiring, error mapping
  prompt.ts      System prompt and model-facing request payload
  schema.ts      JSON Schema builder for structured outputs
  answers.ts     Probability normalization, confidence, answer shaping
  validate.ts    TypeSafe request validation (422s)
  types.ts       TypeSafe wire types and Env
  errors.ts      ApiError -> HTTP response mapping
  util.ts        Object helpers
test/            Vitest suites for validation, schema, payloads, answer math

Development

npm run dev        # local server with .dev.vars
npm test           # unit tests
npm run typecheck  # tsc --noEmit
npm run deploy     # wrangler deploy