diff --git a/.changeset/intent-surface.md b/.changeset/intent-surface.md new file mode 100644 index 0000000..1a08cf6 --- /dev/null +++ b/.changeset/intent-surface.md @@ -0,0 +1,48 @@ +--- +"@webmcp-stack/codegen": minor +--- + +**Generate a small, intent-level surface, and teach your own coding agent the rules.** + +This release changes what `@webmcp-stack/codegen` thinks its job is. It still writes safe, typed tools from your OpenAPI spec. What is new is the shape of the surface: instead of a one-to-one mirror of your routes, you get a few intent-level tools, actions your API split across calls, and journeys for goals that take several steps. + +**Journeys: a real goal, not just an endpoint.** Some things a user asks for cannot be one tool call. Booking a ride is find the destination, set the pickup time, then confirm. A journey is a small file in `src/webmcp/journeys/` that composes the tools you already generated into one flow, with a shared draft and a single confirmed write: + +```ts +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", + steps: { + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: (place) => ({ destination: place }), + provides: ["destination"], + }, + "set-pickup-time": { + description: "Set when the user wants to be picked up.", + input: { type: "object", properties: { pickupAt: { type: "string" } }, required: ["pickupAt"] }, + provides: ["pickupAt"], + }, + }, + submit: { + description: "Request the ride and show the driver's details.", + build: (draft) => draft as RequestRideInput, + run: executeRequestRide, + }, +}); +``` + +`generate` scaffolds the `journey.webmcp.ts` factory next to the runtime, the barrel registers every journey file it finds, and `verify` lints them. The submit gate and the confirmation live in a file the generator owns and rewrites, so a journey file cannot edit them out. + +**One tool for an action your API split in two.** When a begin/end pair like request-upload plus complete-upload is really one action, the generator merges them into a single withheld tool and threads the first response into the second by exact name. A pair it cannot thread is skipped with a note, never guessed. + +**A skill file for your coding agent.** Every run writes `.agents/skills/webmcp-tools/SKILL.md`, where Claude Code, AGENTS.md, and the generic standard already look. It teaches the naming rules, the description budgets, the execute contract, and the journey pattern, so your own agent extends the surface in the shape this tool expects. The repo also ships an eval harness that runs prompts against a fixture and grades the result, so a wording change can be measured instead of guessed at. + +**Your descriptions are never silently shortened.** Chrome's 500 and 150 character budgets are authoring guidance, not rules the browser enforces. Generation composes its own text to fit, keeps your text and your spec's text in full, and `verify` warns on an overrun instead of blocking. A description that reads well no longer gets cut down to pass a counter. + +**Current with the WebMCP spec.** Generated tools carry `title`, the human label native UIs show, destructive tools carry `consequentialHint`, and the `tools` output accepts `exposedTo` to scope which origins your tools are shared with. + +**The CLI no longer calls a model.** `--llm` and `--suggest` are removed, along with the provider flow and its config options. If you used them, the skill file is the replacement: it teaches your own coding agent the rules, and a plain `generate` never touches the network. + +**Docs.** The journeys guide is rewritten around a relatable example, with new guides for why this matters, what to do after you generate, working with your coding agent, a prompt cookbook, choosing between a tool, a group, and a journey, and testing your tools. The docs site publishes `/llms.txt` and `/llms-full.txt` for agents, and the CLI's docs links point at the new home, https://webmcp.souravinsights.com. \ No newline at end of file diff --git a/README.md b/README.md index 1298d7f..1fae9b9 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@
- + @@ -8,15 +8,15 @@

webmcp-stack

The open-source developer stack for WebMCP.

-

Today: codegen. The goal is the whole agent-surface lifecycle in one stack: Generate, Understand, Review, Test, Control, Observe, Secure.

+

Today: codegen. It generates a safe, reviewable agent surface from the contract you already have, and it is built to grow into the rest of the lifecycle.

npm version MIT license

- Docs | + Docs | npm | - Brand | + Brand | About

@@ -35,6 +35,8 @@ npx @webmcp-stack/codegen generate No install, no config for the first run. It detects your app, writes one `.webmcp.ts` file per tool, and adds the registration call to your entry file (additive edits, always reported; if it can't find the entry point, it prints the two lines for you to paste). +It generates from the contract you maintain, not by scanning your app and guessing at intent. A vague spec or schema makes vague tools, so the source is the part worth getting right. If you have an OpenAPI spec or maintained schemas, this is built for you; if not, writing that contract is the first step, and the `schema` source lets you declare tools by hand in the meantime. + ```bash npx @webmcp-stack/codegen dev # local dashboard: browse, edit, toggle, and test tools npx @webmcp-stack/codegen verify # check the tool set against the standard; exits 1 on errors @@ -48,7 +50,9 @@ You can, and it works. What you get back is different every time, and nothing ch - **Decides what agents may do.** Every endpoint is classified read, write, or destructive from the HTTP verb, corrected when the name disagrees (`POST /orders/{id}/cancel` is destructive, `POST /search` is a read). Reads work immediately. Everything else is generated but not registered, so enabling a write is a deliberate edit. Webhooks are skipped, auth and admin endpoints are flagged, and anything it cannot classify starts disabled. - **Asks the user before any mutation.** Write and destructive tools confirm each call with the user (a plain dialog you can replace). The confirmation lives in the generated region of the file, so it cannot be edited away and survive regeneration. -- **Writes the text agents read.** Constraints become sentences ("A number from 30 to 600."), names come from intent (`generate-story`, not `post-trips-trip-id-story-generate`), and every description says what the tool returns. Your own text always wins; machine-written text is marked and flagged in the audit. +- **Writes the text agents read.** Constraints become sentences ("A number from 30 to 600."), names come from intent (`generate-story`, not `post-trips-trip-id-story-generate`), and every description says what the tool returns. Your own text always wins; machine-written text is marked and flagged in the audit. Chrome's character budgets are treated as guidance, not law: machine text is composed to fit, author text is never silently shortened, and `verify` warns on an overrun. +- **Groups actions the API split.** A begin/end pair like `request-upload` plus `uploads/{uploadId}/complete` is one action, not two. The generator detects the pair, threads the first response into the second by exact name, and adds one withheld coarse tool next to the members. A pair it cannot thread is skipped with a note. +- **Scaffolds journeys and an agent skill.** `journey.webmcp.ts` plus `journeys/*.webmcp.ts` express a multi-step flow: a page-scoped draft, one tool per step, and a submit gate that refuses until every step is done and confirms with the human before the real write. The barrel registers them and `verify` lints the files. `.agents/skills/webmcp-tools/SKILL.md` teaches your own coding agent the rules. - **Says what can be trusted.** Free-text outputs get `untrustedContentHint`; PII-looking fields are named in the report and in a comment in the file; mutating tools on authenticated endpoints carry a session warning; `execute` calls your real endpoint, so your own validation still runs. - **Annotates real forms.** A tool that maps to a visible `
` can annotate it in place, so the agent fills the same controls the user sees, and the user reviews and submits the write. @@ -65,11 +69,23 @@ A real tool from a real app: `create-trip`, one of 70+ tools generated for [been */ export const createTripTool = { name: "create-trip", + title: "Create Trip", description: "Create a new trip. Returns the trip.", inputSchema: createTripInputSchema, // title, dates, location, theme, field notes - annotations: { readOnlyHint: false, untrustedContentHint: true }, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, }; +// Journeys and your own code compose this raw caller; executeCreateTrip wraps it +// in the agent-facing result shape. +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "POST", body: { ... }, signal }); + return data; +} + // A write tool, so it is withheld: the registration is generated but commented // out, including the built-in user confirmation, until you enable it: // const confirmed = await requestUserConfirmation( @@ -79,9 +95,8 @@ export const createTripTool = { export async function executeCreateTrip(input: CreateTripInput, signal?: AbortSignal) { return toolDisabled("create-trip.webmcp.ts"); - // Uncomment to go live: - // const data = await callApi("https://api.beenthere.page/v1/trips/", { method: "POST", ... }); - // return toolResult(data); + // Uncomment to go live, and uncomment the registration above: + // return toolResult(await fetchCreateTrip(input, signal)); } ``` @@ -90,9 +105,9 @@ The bar the ecosystem is converging on (Chrome's WebMCP best-practices and tool- | The bar | Today | |---|---| | Verb-first, intent-shaped names, 30 characters or fewer | Enforced on every run | -| Descriptions say what the tool does and when, positively, within 500 characters; every parameter described within 150 | Assembled on every run; length budgets measured by `verify` next | -| Outputs within 1.5K characters; errors that help recovery; user-written content marked `untrustedContentHint` | Errors and annotations enforced; output budget next | -| Exposure as a decision: `readOnlyHint`, registration only where usable, `exposedTo` origin scoping | Annotations and withheld-by-default enforced; `exposedTo` lands as the runtime stabilizes | +| Descriptions say what the tool does and when, positively, within 500 characters; every parameter described within 150 | Assembled on every run; author text is never shortened, and `verify` warns on overruns | +| Outputs within 1.5K characters; errors that help recovery; user-written content marked `untrustedContentHint` | Errors, annotations, and the output cap enforced; the cap lives in the generated runtime | +| Exposure as a decision: `readOnlyHint`, registration only where usable, `exposedTo` origin scoping | Annotations and withheld-by-default enforced; `exposedTo` is a config pass-through to registration | | The human in the loop: visible page effects, confirmation on consequential actions | Confirmation enforced, in the generated region where it can't be edited away; visible-effect hook scaffolded | | No steering: descriptions never instruct the agent or encode flow control | Flagged in the audit | | The schema is not the security boundary; the app still validates at run time | Stated in the output; `execute` calls your real endpoint | @@ -101,24 +116,23 @@ The bar the ecosystem is converging on (Chrome's WebMCP best-practices and tool- ## Changing things later -- **`.webmcp-codegen.json`**: per-tool overrides for descriptions, names, enabled state, fields. Applied last, so your text always wins. +- **`.webmcp-codegen.json`**: per-tool overrides for descriptions, enabled state, and field text. Applied last, so your text always wins. - **Dashboard** (`dev`): edit descriptions, toggle tools, run tools against real endpoints. Writes back to the overrides file. - **Audit**: problems reported in plain language every run: missing descriptions, mislabeled verbs, PII in outputs, agent-instruction smells, oversized surfaces. Errors block file writing; warnings do not. - **`verify`**: the standard, checked locally. Built for CI. -- **LLM help if you want it**: `--suggest` proposes tools worth declaring, `--llm` drafts descriptions. Advisory only: it never classifies risk, never changes exit codes, and a plain `generate` never makes a network call. +- **Skill file**: `.agents/skills/webmcp-tools/SKILL.md` teaches your own coding agent the rules (naming, description budgets, journeys). Regenerated on every `generate`; add your own skill directory to stack project-specific rules on top. ## Where this is going Codegen first: - **More sources.** OpenAPI and validation schemas today, tRPC on the list. The rule holds: contracts, not codebases, and the CLI never scans app code. -- **Journeys.** One tool per CRUD operation is rarely the right shape; agents do better with a few coarse, intent-level tools. Declare a flow once (draft store, step tools, a submit gate), run it in the dashboard, review the generated Mermaid diagram. -- **The dashboard becomes the review surface.** LLM suggestions arrive as an accept/reject queue that writes to the overrides file. In progress now. -- **Evals for the LLM layer.** A prompt change becomes a visible regression, not a vibe. +- **The dashboard becomes the review surface.** A browse-and-score report over the generated surface, with the editing UI kept for the overrides it writes. Parked until the audit package exists. +- **Skill-file evals.** Shipped at `packages/codegen/evals/skill/`: a prompt set, a generated fixture, deterministic graders, and a control case that runs the sharpest prompt with the skill removed, so you can tell when a model has absorbed the rules. Then the stack around it: **audit** (point it at a URL, get a report on the surface a visiting agent would find) and **telemetry** (how agents actually use your tools). The goal is one stack where each tool covers one stage of the lifecycle and they compound. The bet underneath: websites are growing an agent-facing surface the way they grew APIs, and that surface needs the same kind of tooling, with higher stakes, because the caller is a model acting as your user, inside your page. -The guarantees hold through all of it: the repo pins the WebMCP draft it targets and watches for spec drift; your overrides, execute bodies, and review decisions survive every regeneration; breaking changes print the exact fix. Deterministic where possible, LLM where useful, honest always. +The guarantees hold through all of it: the repo pins the WebMCP draft it targets and watches for spec drift; your overrides, execute bodies, and review decisions survive every regeneration; breaking changes print the exact fix. Deterministic where it can be, honest always. ## Principles @@ -134,7 +148,7 @@ The guarantees hold through all of it: the repo pins the WebMCP draft it targets | `packages/codegen` | `@webmcp-stack/codegen` | The CLI and the generation pipeline: sources (OpenAPI, validation schemas), outputs, the safety audit, the dev dashboard. | | `examples/openapi-petstore` | private | Example app with tools generated from the Petstore OpenAPI spec. | | `site/` | private | Landing page and documentation (Next.js + Fumadocs). | -| `docs/` | - | Design specs (`specs/`), decision notes (`notes/`), and [what this project is](./docs/about.md). | +| `docs/` | - | [About](./docs/about.md), design specs (`specs/`), decision notes (`notes/`), research (`research/`), and reviews (`reviews/`). | | `scripts/` | - | Committed git hooks (lint on commit, lint + typecheck + test before push). | | `brand/` | - | Logo and brand assets. | @@ -152,7 +166,7 @@ pnpm lint:fix ``` ```bash -pnpm --filter openapi-petstore dev # example app +pnpm --filter example-openapi-petstore dev # example app pnpm --filter site dev # landing page & docs on :3001 ``` diff --git a/docs/research/2026-09-07-what-to-double-down-on.md b/docs/research/2026-09-07-what-to-double-down-on.md new file mode 100644 index 0000000..79c9802 --- /dev/null +++ b/docs/research/2026-09-07-what-to-double-down-on.md @@ -0,0 +1,351 @@ +# What we build, and how + +**Date:** 2026-09-07 + +Agents are the users now. Someone runs codegen once to scaffold their tools, +then asks their own model to improve them. We stop building anything the +user's own agent already does. What's left: make the existing surface more +reliable, add the two valuable things (skill file, journeys), and remove +what's unnecessary. Parked for later: the audit package and the dashboard +rework. + +## codegen — the scaffold + +The front door. Three jobs: + +**1. Bake in the safety defaults nobody applies by hand.** Chrome's guidance +sets hard limits and per-tool flags; across 40 endpoints everyone forgets +some. The generator applies them to every tool, every run: + +- Names max 30 characters: enforced at generation and checked by `verify` + (shipped in 0.7). Still to build: tool descriptions max 500 and parameter + descriptions max 150 — composed to fit at generation, trimmed at a + sentence end and marked machine-written when inherited spec text + overflows, and measured by `verify` so CI gates on them. +- Tool outputs max 1.5K characters (to build). You can't check this before + the tool runs, so the generated code passes every result through a helper + that truncates it — and that helper sits in the part of the file only the + generator edits, so nobody deletes it by accident. +- Read tools get marked read-only automatically (shipped). Outputs + containing free-text fields get marked as untrusted user content, + including fields nested inside arrays, objects, and nullable unions + (shipped in 0.7). Destructive tools get marked consequential + (`consequentialHint`, spec annotation — to ship). Origin scoping + (`exposedTo`) ships as a config option — the spec API exists now + (to ship; see docs/research/2026-09-10-spec-sync.md). +- Write and destructive tools ask the user to confirm each call, in the + generator-owned part of the file (shipped). + +**2. Say plainly that the schema is not the security boundary.** A nice JSON +Schema doesn't make a call safe — the app still has to validate types, +permissions, and state when the tool runs. So `execute` always calls the +app's real endpoint, never a generated shortcut around it, and every +generated file carries one comment line saying exactly this. We never +describe the tool as a security feature. The phrase is "scaffolding with +safe defaults." + +**3. Generate intent-level tools, and call the output a draft.** One tool per +endpoint is the wrong shape — agents do better with a few coarse tools like +"search flights" than with a mirror of your CRUD API. So generation gets a +grouping step: cluster endpoints by resource, emit the obvious coarse tools, +leave mutations commented out. The report shows the grouping as a proposal; +the user or their agent adjusts it through the overrides file; regeneration +keeps their decisions. Without this, codegen is just a 1:1 mapper with +extra steps — this is what makes it not that. + +**Grouping and journeys are not the same thing.** Grouping *merges*: several +endpoints that are one action split by API shape become one tool — +beenthere's `create-media-request-upload` + `complete-media-upload` are an +upload handshake no agent should see; one `upload-media` tool wraps both +calls. It happens at generate time, from the spec, no human input. Journeys +*chain*: several different decisions with shared state and one guarded +submit (`document-trip`). They're written after generation, by the user's +agent, because the flow lives in the product. One-line test: same action +split across calls → group it; different decisions along the way → journey. +Grouping shrinks the standalone surface; journeys make dangerous writes +reachable under a gate — and tools a journey covers get absorbed into it +(flow-only tools never register standalone), so the total registered surface +goes down, not up. `verify`'s surface budget counts journey tools too, so +this is enforced, not aspirational. + +## journeys — one file we ship, small files the user's agent writes + +**Our code is exactly one file: `journey.webmcp.ts`.** When `generate` runs, +it writes this file into the user's repo next to `runtime.webmcp.ts`, under +the same contract as the runtime file: fully ours, regenerated on every run, +never hand-edited. ~120 lines, no dependencies beyond the runtime helpers +that already ship. The complete, current copy lives at +`packages/codegen/assets/journey.webmcp.ts` — that file is the review +artifact; this section explains it. + +The contract it exposes — this is the part that matters: + +```ts +// A step backed by an existing generated tool: inherits its description and +// input schema, calls the raw caller the generated file exports. You write +// only what's new — which slice of the result lands in the draft. +interface ToolStep { + tool: { description?: string; inputSchema?: Json }; // e.g. getAutocompleteTool + call: (input, signal, draft) => Promise; // e.g. fetchGetAutocomplete + store: (result: unknown) => Json; // what lands in the draft + provides: string[]; // draft fields submit waits for + description?: string; // override; default: the tool's + input?: Json; // override; default: the tool's +} + +// A step with no backend call — it just collects input into the draft +// ("set the title and dates"). With a run, it can do work first. +interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input, signal, draft) => Promise; // default: store the input +} + +interface JourneyDef { + name: string; // "document-trip" — step tools derive from it + goal: string; // the one sentence every step repeats to the agent + steps: Record; // ToolStep | FreeStep + submit: { + description: string; // what the human confirms + build: (draft) => unknown; // assemble the real tool's input + run: (input: never, signal?) => Promise; // the real tool's execute + }; +} +``` + +Reading that code, the three pieces are: + +1. **The draft** — `let draft = {}`, one plain object per page load. Steps + write into it; the submit reads from it. It dies with the page: a + half-finished journey doesn't survive a reload, which is what you want. +2. **Step tools** — `registerSteps`: ordinary registered WebMCP tools named + `-` (`document-trip-search-places`). A step's `execute` + stores its result into the draft and replies with what's still missing, + so the agent always knows the next move. +3. **The submit gate** — `registerSubmit`: one more tool, `-submit`, + whose `execute` does four things in order: refuse with the missing list → + ask the human to confirm → run the real tool's `execute` with + `build(draft)` → clear the draft. No way around it, because the real + input only exists inside `build(draft)`. + +**What the CLI does around that one file — three mechanical jobs:** + +1. **Copy it in.** The same code path that already scaffolds + `runtime.webmcp.ts` on every `generate`. One template change rides along: + generated tool files also export the raw caller (`fetchGetAutocomplete` — + just the `callApi` line, unwrapped), because journeys compose raw data, + not agent-shaped results. +2. **Register journeys on page load.** The generated `index.ts` — the file + that today exports `registerAllTools()` — also imports every export of + `/journeys/*.webmcp.ts` and calls its `.register()`. Dropping + a new journey file into that folder and re-running generate is the + whole wiring story. +3. **Check journey files in `verify`.** Submit gate present, every step + described within budget, step count ≤5, no PII-shaped draft field leaking + into a step's output. + +Plus one report line at generate time when endpoints cluster like a flow +("3 stamp endpoints and a per-trip eligibility read look like one flow — +declare a journey?"). A hint, never an auto-generation. + +**What we never write: the journey definitions themselves.** Every +`/journeys/*.webmcp.ts` file is the user's agent's code, written with the +skill file's guidance — only the product side knows the flow. What codegen +can't do: read an OpenAPI spec and discover that "create a trip" is really +search → set details → create → open the editor. That knowledge lives in +the product, not the API contract. A CLI guessing flows produces plausible +garbage. + +Two real examples of those user-side files, from beenthere's generated +surface: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + "search-places": { + tool: getAutocompleteTool, // schema + description inherited + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: pickFrom(places) }), // the resolved pick + provides: ["locationObject"], + }, + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" }, startDate: { type: "string" }, endDate: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, + run: executeCreateTrip, // POST /v1/trips/ + }, +}); +``` + +Why this journey exists: `create-trip`'s input has a `locationObject` with 7 +required fields (`placeId`, `fullAddress`, `country`, ...) that an agent +cannot invent — it comes from `get-autocomplete`. Called cold, the agent +hallucinates a place and fails server-side validation. The journey's draft +carries the resolved place from step 1 into the submit, so the hard input is +real by construction. + +Why it's named `document-trip` and not `plan-trip`: beenthere is a journal +for trips you've *been on*, not a planner. Naming it `plan-trip` would pass +every mechanical check — verb-first, short, maps to a real endpoint — and +still be wrong. That class of mistake is the skill file's job; see below. + +```ts +// src/webmcp/journeys/collect-stamp.webmcp.ts +export const collectStamp = createJourney({ + name: "collect-stamp", + goal: "Generate a stamp for a city the user has a qualifying trip for", + steps: { + "pick-trip": { + description: "Pick the trip to base the stamp on.", + input: { type: "object", properties: { tripId: { type: "string" } }, required: ["tripId"] }, + provides: ["tripId"], + }, + "check-eligibility": { + tool: getTripStampEligibilityTool, + input: { type: "object", properties: {} }, // the agent provides nothing + call: (_input, signal, draft) => fetchGetTripStampEligibility({ tripId: String(draft.tripId) }, signal), + store: (eligibility) => ({ eligibility }), + provides: ["eligibility"], + }, + }, + submit: { + description: "Generate the stamp (a paid generation call).", + build: (draft) => ({ name: cityFrom(draft.eligibility) }), + run: executeGenerateStamp, // POST /v1/stamps/generate + }, +}); +``` + +Why this one exists: `generate-stamp` is a paid generation call, and a stamp +only makes sense for a trip that qualifies. Without the journey shape, an +agent generates stamps for cities the user never visited; with it, the +eligibility check gates the spend, and the submit reuses the trip from the +draft. (A step's `call` receives the draft so far — that's how +`check-eligibility` reads the `tripId` that `pick-trip` stored.) + +No JSON config DSL — the declaration references the user's schemas and +tools, so it's code anyway; a second config language is weight for nothing. +The skill file teaches the user's agent how to turn the report's cluster +hint into the files above. + +## skill file — the rules, sitting in the user's repo + +The actual file is written and reviewable: +`packages/codegen/assets/skill/SKILL.md` (standard skill format — YAML +frontmatter with a `description` written as the trigger, then the rules). +The generator drops it into the user's repo on first run, next to the tools. +It teaches whatever agent the user runs: the naming rules, description +budgets, exposure decisions, the execute contract, how to edit generated +files without fighting regeneration, and the journey pattern above. + +This is how the rules reach the user's model when it edits their tools — no +API keys, no network calls, works even when the prompt is lazy. Their agent +knows their product, which our CLI never can, so this is also where good +grouping decisions actually happen. + +The hardest rule it teaches: **before naming an intent-level tool or +journey, understand the product** — what it is, who uses it, when in the +user's life the action happens. If that's not written down anywhere, the +agent asks the user. A wrong-tense or wrong-role name (`plan-trip` on a +memories product) passes every mechanical check and still ships wrong; this +is the one failure no deterministic check can catch, so the discipline lives +in the file the agent reads while it works. + +### The eval plan + +Following the two references the user pointed at +([OpenAI](https://developers.openai.com/blog/eval-skills), +[philschmid](https://www.philschmid.de/testing-skills)): an eval is +prompt → captured run → checks → score. Concretely for this skill: + +1. **Prompt set** — 10–20 cases in `evals/skill/`, each a situation plus its + own expected checks. Categories: trigger tests ("add a webmcp tool for + X" should activate the skill), core tasks (write a tool from a schema, + improve a generated description, enable a withheld write tool), the + journey fixture (below), and negative controls (an unrelated coding + prompt — the skill must not trigger). +2. **The sharpest fixture** — a copy of beenthere's generated surface with + the prompt "write the trip journey." The right answer is retrospective: + name matches `document|record|log`, and `/plan|book/` is an automatic + fail. Regex-checkable, no judge needed. +3. **Harness** — run the agent CLI headlessly (`codex exec --json` or + equivalent) in a clean copy of the fixture repo per case, 3–5 trials per + case since behavior is nondeterministic. Grade outcomes, not paths. +4. **Deterministic checks first** — produced file parses; name verb-first + and ≤30 chars; description ≤500 and says what returns; writes keep the + confirmation call; `execute` calls the real endpoint; nothing edited + above the generated marker; journeys use `createJourney` with a submit + gate. +5. **LLM rubric second, selectively** — for what regex can't grade (is the + tool intent-shaped? did the agent ask about product context?), a second + pass constrained to a structured schema (`overall_pass`, per-check + results), so scores diff across runs. +6. **Operate like tests** — every real failure becomes a new case; once a + case hits ~100% it graduates into a regression suite; run the suite with + the skill unloaded occasionally — if everything still passes, the model + absorbed the skill and we retire it. + +## Out of scope for now + +Two pieces from earlier versions of this direction are parked — real, but +later, after the reliability work above lands: + +- **The audit package** (`@webmcp-stack/audit`): extracting the checks into + their own package with `verify` as a thin wrapper, plus the paste-a-URL + audit. `verify` keeps growing inside codegen for now; extraction happens + when the URL-audit product actually starts. +- **The dashboard as a report**: browse + scorecard + model-based review, + with the editing UI removed. Not now. (Removing the LLM layer is *not* + parked — that's deletion, and deletion is in scope.) + +## What gets removed + +- **The LLM suggestion layer** (`--suggest`, `--llm`, `src/llm.ts`). Frozen + the day the skill file ships, deleted a release later. The user's own + model does this better. +- **The big roadmap** (Test, Control, Observe, Secure). Narrowed to two + jobs: write the rules down, check tools against them. + +## Order of work + +Already shipped (0.7–0.8.2, current on npm): withheld-by-default writes, +confirmation gates, the naming rules, descriptions that say what a tool +returns, nested untrusted-content marking, and `verify` with its scorecard +and `--url` check. + +Shipped 2026-09-10 on the journeys feature branch (feat/journeys): + +1. ~~Spec sync quick wins~~ — `title` emitted everywhere (tools and journey + gates), `consequentialHint` on destructive tools, `exposedTo` config + pass-through on the tools output, Chrome 149/150 version fixes. +2. ~~Budgets~~ — generation composes within 500/150 (fitBudget, sentence + cuts), verify's new error-level Budgets check measures the final text, + the runtime caps every `toolResult` at ~1.5K with a truncation notice. +3. ~~Skill file~~ — scaffolded at `.agents/skills/webmcp-tools/SKILL.md` + (cross-client location, Claude Code included), regenerated like the + runtime; eval harness at `packages/codegen/evals/skill/` with the + beenthere-lite fixture, nine cases, deterministic graders, and a passing + self-test (`node run.mjs --selftest`). +4. ~~Journeys~~ — `journey.webmcp.ts` scaffolded next to the runtime, the + barrel registers every `/journeys/*.webmcp.ts` it finds, + endpoint-backed tools emit the `fetchX` raw caller that steps compose, + and verify lints journey files (gate, budgets, step count, no raw fetch). +5. ~~Grouping~~ — `groupHandshakes` merges POST handshake pairs + (request-upload + complete-upload → upload-media) with exact-name + threading only; the merged tool is a withheld draft, members untouched, + the CLI prints the proposal line. +6. ~~LLM layer~~ — deleted: `--llm`, `--suggest`, the config options, the + provider flow, ~1,050 lines gone. The skill file is how rules reach + models now. + +Parked (out of scope for now): the audit-package extraction, the +dashboard-as-report rework, and the paste-a-URL audit. diff --git a/docs/research/2026-09-10-spec-sync.md b/docs/research/2026-09-10-spec-sync.md new file mode 100644 index 0000000..bb6dfd1 --- /dev/null +++ b/docs/research/2026-09-10-spec-sync.md @@ -0,0 +1,90 @@ +# WebMCP spec sync — 2026-09-10 + +What changed in webmachinelearning/webmcp since our last read, what it means +for us, and what we do about it. Read against the local clone of +webmachinelearning/webmcp at 97da8f5 (main, current as of today), the spec +(index.bs), and Chrome's two guides (best practices, secure tools). + +Bottom line: we've been tracking well. Three small API additions ship with the +next release, then we're back on the 09-07 plan. + +## Compliant already — no action + +- **Character budgets (500/150/30, 1.5K output).** Still the numbers in + Chrome's secure-tools guide. Our planned verify thresholds match current + guidance. +- **Name constraints.** Spec rejects names over 128 chars or containing + anything outside `[A-Za-z0-9._-]` with `InvalidStateError`. Our 30-char + kebab-case names comply. +- **Annotations.** `readOnlyHint` (auto on reads) and `untrustedContentHint` + (recursive free-text detection, shipped in 0.7) are exactly the two the + secure-tools guide recommends. +- **executeTool.** Signature is now + `executeTool(RegisteredTool, optional any inputObject, options?)` returning + a stringified result. Our dev-dashboard testing path is compatible. +- **Withheld-by-default + confirmation.** The spec's security section and + Chrome's guides push exactly the posture we already generate: registered + tools are not visible to other origins by default; consequential actions + should be gated. + +## Gaps — small, ship next release + +1. **`title` — new member we don't emit.** `ModelContextTool.title` + (USVString) is a human-facing label for native UIs (recommended + localized). RegisteredTool now carries it back. Cheap: generate + `title: "List Trips"` from the tool name. +2. **`consequentialHint` — new third annotation we don't emit.** "Executing + will result in consequential actions — significant, real-world, + non-reversible (booking a flight, transferring money)." Auto-set it on + tools our classifier marks destructive; the report can flag borderline + mutations for review. +3. **`exposedTo` has landed — our "flag it until the runtime supports it" + note is now actionable.** `registerTool(tool, { exposedTo: [...origins] })` + is in the spec (rejects non-trustworthy origins with `SecurityError`) and + documented in Chrome's secure-tools guide. Add a config option that passes + through to registration, exactly as planned. +4. **Version claims in our docs are stale.** We say Chrome 146+; the origin + trial is live in **Chrome 149** (Edge 150), and the + `#enable-webmcp-testing` flag is the local-development path. Three files: + `packages/codegen/README.md`, `site/content/docs/devtools.mdx`, + `site/content/docs/guides.mdx`. + +Implementation note for 1–3: we ship our own minimal `ModelContext` types in +the generated runtime (structural typing, no webmcp-types dependency), so the +fixes are template edits — `title?` and `consequentialHint?` on the tool +definition, `exposedTo?` on registration options. Generated code stays +compatible with browsers that don't know the new members yet. + +## Watch items — not in the spec yet, do not build on them + +- **`requestUserInteraction()`** — Chrome's secure-tools guide references it + for native user prompting/elicitation at tool-execution time, but it is not + in index.bs today. If it lands, it replaces our `confirm()`-based + `requestUserConfirmation` helper with a browser-mediated prompt — a clean + swap since confirmation calls live in the generated region. Also the + headless story (#296 made headless a goal): `window.confirm` has no headless + UX; a native API solves that too. +- **Skills integration (#161)** — still an open question in the spec. If a + native skill/flow construct ships, our journeys adopt it: the factory is one + file and the convention (prefix names, "Part of…" suffix) already mirrors + what a skill construct would formalize. Keep journeys as-is until then. +- **`outputSchema` (#9)** and **native input/output validation (#92)** — open + issues. If native validation lands, our generated schemas get checked by the + browser; nothing for us to change now, but it would make the "schema is not + the security boundary" guidance more literal. +- **Declarative (form-based) API** — real and documented + (declarative-api-explainer.md); forms gain `toolname`/`tooldescription`. + Awareness only: codegen is imperative-first. If users ask for form coverage, + that's a separate feature. +- **Service workers explainer** — background tool discovery without an open + page. Early; awareness only. + +## One philosophical note, honestly recorded + +Chrome's best practices say "trust the agent — don't enforce rigid step-by-step +procedural chains." Our journeys sequence steps, but the enforcement lives in +tool *outputs* (refusals, "still needed: X" replies), never in description +prose — the thing they warn against. The step descriptions only carry goal +context ("Part of document-trip: …"). We sit on the right side of the line, +but it's a line: if a future spec version formalizes flows (likely via #161), +we converge to it. diff --git a/docs/reviews/2026-09-11-pr-4-intent-surface.md b/docs/reviews/2026-09-11-pr-4-intent-surface.md new file mode 100644 index 0000000..03cbbb8 --- /dev/null +++ b/docs/reviews/2026-09-11-pr-4-intent-surface.md @@ -0,0 +1,268 @@ +# PR #4 review: The intent surface + +**PR:** https://github.com/SouravInsights/webmcp-stack/pull/4 +**Branch:** `feat/intent-tools` against `main` +**Reviewed:** 2026-09-11 + +## The short version + +The hard parts are built well. Grouping the upload handshake only when the rules +are exact is the right call. Putting the submit gate in a generator-owned file +means a journey can't edit the gate out. Tests pass, types pass, and the spec +sync is real. + +The one thing to settle before touching code: codegen treats Chrome's character +advice as if it were a law the browser enforces. It isn't. Once that's clear, the +right fixes get easier, and one of them (the trimmer) stops being a straight bug +fix and becomes a design decision. + +## First: are those character limits actually rules? + +This matters, so I checked the spec itself rather than trusting the summary in +the code comments. + +**What the browser actually rejects.** In `index.bs`, registering a tool fails +only for these reasons (lines 668-678): + +- a tool with that name is already registered +- the name or the description is an empty string +- the name is longer than 128 characters, or contains anything other than + letters, digits, `_`, `-`, or `.` +- the `inputSchema` can't be serialized to JSON + +**There is no description length rule in the spec.** Not 150, not 500. Nothing. +The spec's own roadmap text says the opposite: the 128-char name is the only +"nominal size restriction," and "further work is needed to evaluate the right +size limits for titles, names, and other inputs" (line 1816, issue #73). + +**Where 150/500/1.5K come from.** The spec README's Best Practices section points +at two Chrome guides (`best-practices` and `secure-tools`) for the detailed +numbers. Those guides are the source. They are authoring advice, and exceeding +them does not error. Nothing rejects a 600-character description. + +So we have three different things that were being treated as one: + +1. **Hard rules the browser enforces.** Name length and charset, non-empty name + and description. Break these and registration fails. Codegen must never break + these, and today it doesn't. +2. **Chrome's authoring guidance.** 30-char names, 150-char parameter text, + 500-char tool text, roughly 1.5K of output. These help the model pick and + call tools, and keep context small. Overrunning them is a quality smell, not + a failure. +3. **This project's own policy.** Generation truncates to those numbers and + `verify` marks overruns as errors. That is a choice codegen made. It's a + defensible choice, but it's a policy, not a law. + +## What should codegen do about it? + +The goal is a tool that writes good descriptions, not a tool that wins a +character-count contest. A few conclusions: + +**Never silently shorten text a human or a spec wrote.** That's the real problem +with the current behavior. The developer (or their spec) had a precise sentence; +the generator quietly chops it. The generated region can't be hand-edited, so the +only way back is an override file most people won't know about. If something has +to give, keep the full text and tell them it's long. Silently changing meaning is +the worst of the options. + +**When codegen writes the description itself, aim at the budget by construction.** +The generator controls its own phrasing, so it can compose text that naturally +fits, instead of composing something too long and then cutting it. This is where +being budget-aware actually belongs. + +**Report overruns, don't block on them.** A 160-character description is not a +broken tool. Make it a warning in `verify` that says the real length, so a +developer or their coding agent can decide whether to tighten it. Keep `error` +for the things that genuinely break: a spec-invalid name, an empty name or +description, an unserializable schema. + +**Make the budgets configurable.** Teams that want Chrome's exact numbers as a +CI gate can turn on strict mode. The default shouldn't fail a build over a +sentence that reads fine. + +**Watch how truncation and `verify` fight each other.** Today `verify` warns when +a description doesn't say what the tool returns. But the most common reason a +description overflows is that it has a second sentence saying what it returns, +and the trimmer cuts exactly that sentence off. So the generator can create the +problem it then complains about. Whatever is decided for budgets has to be +decided for both generation and `verify` at the same time, or they'll contradict +each other. + +**Keep the output cap as the exception.** The ~1.5K limit on tool results is +different: it protects the model's context at runtime, the helper already adds a +visible "truncated" notice, and cutting a giant JSON blob doesn't destroy +meaning the way cutting a sentence does. Truncation is the right tool there. + +## The problems, in plain terms + +### 1. The description trimmer has a real bug, but the deeper issue is the trimming + +Two separate things here. + +The bug: a parameter description is supposed to be cut to 150 characters. If the +text is one long unbroken string (a URL, a token, an ID), the trimmer keeps all +150 and then adds an ellipsis, landing at 151. I reproduced it. Since `verify` +treats 150 as an error, the generator can produce text its own checker rejects, +and the developer can't fix it in the generated region. If we keep any trimming, +that line needs to reserve room for the ellipsis. + +The deeper issue: per the section above, 150 is Chrome's advice, and cutting a +description at a character boundary is a blunt way to honor it. Before fixing +the off-by-one, decide whether silent truncation of author text is the behavior +we want at all. My suggestion is no: compose short, warn on long. But if the +team decides to keep truncating, fix the one line and add a test for the +no-space case. + +### 2. Two safety promises the docs make, that the code doesn't keep + +Both are in the new journey FAQ (and repeated in the direction doc). + +**"Journey tools count toward the surface limit, and CI fails when the list +gets too big."** Neither half is true. The surface check only looks at generated +endpoint tools; journey step tools are built at runtime and never reach it. And +going over the limit is a warning, so it doesn't fail CI. + +**"A tool that only makes sense inside a journey won't register on its own, so +the journey shrinks your surface."** Nothing implements this. When +`document-trip` gets its place data from `get-autocomplete`, that tool still +registers on its own, because reads register by default. It's a convention the +developer has to follow, presented in the docs as if the system enforces it. + +Why it matters: a reader will trust that the audit is watching this and stop +watching it themselves, then ship forty tools while `verify` stays quiet. + +The other AI's suggested resolution is reasonable: count the journey tools (steps +plus one submit per file, which you can already compute from the linted files) +and make over-limit a real error, then reword the absorption sentence to say it's +a convention. Auto-hiding a standalone read that a journey fully absorbs can wait +for later. + +### 3. A journey step is labeled "read-only" even when it might write + +Every journey step is registered as read-only, whatever it actually does. A +step's `call` is just user code. It can call the real create-trip caller directly +instead of going through the submit gate. When that happens, the tool is +advertised to the agent as safe, the write goes through with no confirmation, +and the audit says nothing. + +The other AI pushed back on my first framing, and it was right: a regex can't be +the answer here, because the deepest bypass is editing the owned half of +`create-trip.webmcp.ts`, and no lint will catch that. That's a separate integrity +feature. + +But there are two different cases and it's worth keeping them apart: + +- **A step that calls a generated write caller** (`fetchCreateTrip` or + `executeCreateTrip`). No editing required, just a coding agent taking a + shortcut. This is reachable today, and a lint *can* flag it, because the caller + is generated and its tool is already classified as a write. Worth doing as a + tripwire, not a guarantee. +- **A hand-edited owned region.** Not catchable by any lint. Out of scope here. + +The honest fix for the actual defect is to stop claiming the steps are read-only +when we can't prove it, and to write down that a journey is only as safe as the +person who wrote it. The lint tripwire is a bonus. + +### 4. Journeys can silently fail to register + +Generated tools ask the browser for the WebMCP API at the moment they register. +Journeys ask once, when the file first loads, and remember the answer. If the API +appears in between, the tools register and the journeys don't, with no error. +Fix is small: look it up at registration time, the same way the tools do. + +### 5. Journey step descriptions can go over the limit + +The factory takes a description already trimmed to 500 and appends +`Part of "document-trip": ...`, so the final text can be over. `verify` only +measures the text in the journey file, not what the factory builds at runtime. +This is the same policy question as #1: if budgets are guidance, this is a +warning at most; if they're hard, trim the base before appending so the composed +text fits. + +## Smaller things, worth a look but not blockers + +- **The root `README.md` still sells the two flags this PR deletes.** It says + `--suggest` and `--llm` exist. The package README and the docs site were + updated; the root one was missed. +- **`docs/specs/journeys.md` describes a design that was never built.** It + describes declaring journeys in a config block. What shipped is TypeScript + files. The spec is marked "not yet implemented", but the feature now exists in + a different shape, so the next person reads the wrong plan. +- **The old LLM layer is still all over `docs/specs/`.** A one-line "removed in + 0.9" note would stop someone rebuilding it. +- **No changeset.** The repo's own rule is to add one for user-facing changes, + and this removes two CLI flags, adds files, and changes what the package + publishes. +- **Small cleanups:** dead code in `evals/skill/run.mjs`, and one Biome warning + from a test string in `group.test.ts`. + +## What's genuinely good here + +- **Grouping is disciplined.** It merges only when four exact rules line up, and + skips anything fuzzy with a note. The tests pin the real beenthere pair, the + unthreadable pair, the cross-resource pair, and a GET negative. +- **The gate sits in owned code.** For journeys that use the factory as intended, + there's genuinely no way around the confirmation. +- **Budgets have one source of truth.** Generation and `verify` read the same + constants. That's the right structure even if the policy around it needs a + decision. +- **The evals grade files, not vibes.** A self-test for the graders and a + "skill removed" control case is more rigor than most features get. +- **The spec sync is honest.** The watch items really aren't in the spec. + +## If I were merging this + +1. Decide the budget policy first: guidance (warn) or hard limit (error). Then + make generation and `verify` agree. My vote is guidance, composed short, with + warnings instead of silent cuts. +2. If truncation stays, fix the off-by-one so it can never return 151. +3. Correct the two doc promises about surface counting and tool absorption. +4. Stop labeling journey steps read-only, and add the best-effort write-caller + tripwire. +5. Look up the WebMCP API at registration time. +6. Update the root README and add a changeset. + +## Post-review discovery: the generated runtime did not parse + +While typechecking the copied journey factory, I found something worse than +everything above. The 1.5K output cap this PR added writes a truncation notice: + +```ts +const TRUNCATED_NOTICE = + "\n… [truncated to fit the 1.5K output budget]"; +``` + +That `\n` sits inside the outer template literal that builds the runtime file, +so the emitted file contains a real newline inside a double-quoted string. +Every generated `runtime.webmcp.ts` is a syntax error, which means no generated +tool compiles at all. The committed eval fixture carried the same broken file, +and because the graders only match text and never compile, it passed every +check. This was invisible until something tried to parse the output. + +## What was implemented + +All of the above is now in the branch, plus the runtime fix: + +1. **Budget policy is guidance, not law.** `describe.ts` no longer silently + shortens author or spec text. Machine-drafted text is composed to fit; + author text is kept in full. `verify` reports overruns as warnings, for both + tools and journey files. Tests updated to lock the new behavior. +2. **The trimmer off-by-one is fixed** and covered by a test with an unbroken + token. +3. **Journey steps no longer claim to be read-only by default.** A tool-backed + step inherits the composed tool's `readOnlyHint`; a free step with no `run` + is read-only; anything else defaults to not-read-only, with an opt-in + `readOnly` override. +4. **The WebMCP API is resolved at registration time** in the journey factory, + matching generated tools. +5. **Journey step descriptions are composed within budget**, keeping the + journey-name tag and fitting the base when the goal would overflow. +6. **The surface check counts journey tools** (one per step plus the submit + gate) via `countJourneyTools`, so the docs' promise is now true. +7. **The generated runtime parses again.** `generated-code.test.ts` runs every + output through the TypeScript compiler to catch escaping mistakes. +8. **Docs corrected:** the FAQ's surface and absorption claims, the safety + bullet on the journeys page, the skill file's step guidance, the root README + (removed the deleted LLM flags), and a status note on `docs/specs/journeys.md`. +9. **A changeset was added.** The fixture was regenerated and the skill copies + were synced. `biome check` is clean, typecheck passes, and 229 tests pass. diff --git a/docs/specs/journeys.md b/docs/specs/journeys.md index cb5736f..0cc98be 100644 --- a/docs/specs/journeys.md +++ b/docs/specs/journeys.md @@ -8,7 +8,12 @@ > tools that pipeline produces). Evidence: > `docs/research/2026-09-02-chrome-webmcp-docs-analysis.md`. > -> Status: spec, not yet implemented. Ships after the generation pipeline (0.5). +> Status: shipped in 0.9, but in a different shape than this early spec. The +> config-declaration design below (a `journeys:` block) was never built. +> What ships: TypeScript files under the tools output's `journeys/` folder that +> import `createJourney` from the generator-owned `journey.webmcp.ts`, plus a +> `verify` lint over those files. Kept for decision history; treat the config +> block below as superseded. ## The problem diff --git a/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md b/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md new file mode 100644 index 0000000..356ca92 --- /dev/null +++ b/examples/openapi-petstore/.agents/skills/webmcp-tools/SKILL.md @@ -0,0 +1,130 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + + + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default - follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch - it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") - that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld - generated but not registered - until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` - the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick + provides: ["locationObject"], + }, + // Freeform step: no backend call - collects input straight into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2-5 steps. More means two journeys. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file - `verify` flags it. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/examples/openapi-petstore/README.md b/examples/openapi-petstore/README.md index 7593a4e..11b8409 100644 --- a/examples/openapi-petstore/README.md +++ b/examples/openapi-petstore/README.md @@ -1,4 +1,4 @@ -# Example: OpenAPI → WebMCP tools +# Example: OpenAPI -> WebMCP tools The smallest possible end-to-end demo of `webmcp-codegen`: a trimmed Petstore OpenAPI spec, one config file, and the generated output (committed so you can @@ -30,7 +30,7 @@ node ../../packages/codegen/dist/cli.js generate ## The regeneration promise -Edit the `throw new Error("Not implemented…")` in any tool's `execute()` to +Edit the `throw new Error("Not implemented...")` in any tool's `execute()` to return something real, then change a description in `openapi.yaml` and re-run `generate`. The description updates; your `execute()` is untouched. That split is the point of the tool. diff --git a/examples/openapi-petstore/openapi.yaml b/examples/openapi-petstore/openapi.yaml index 25a60ba..b664068 100644 --- a/examples/openapi-petstore/openapi.yaml +++ b/examples/openapi-petstore/openapi.yaml @@ -72,7 +72,7 @@ paths: /pets/{id}/adopt: post: operationId: adoptPet - summary: Adopt a pet — this finalizes the adoption paperwork + summary: Adopt a pet - this finalizes the adoption paperwork parameters: - name: id in: path diff --git a/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts index 9c62766..33dc471 100644 --- a/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/adopt-pet.webmcp.ts @@ -1,8 +1,8 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** - * Adopt a pet — this finalizes the adoption paperwork. Returns the pet. + * Adopt a pet - this finalizes the adoption paperwork. Returns the pet. * * Source: POST /pets/{id}/adopt (openapi). Risk: write-confirm. * Starts withheld: not registered until you enable it (see registerAdoptPet below). @@ -15,7 +15,7 @@ export const adoptPetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const adoptPetHints = {"readOnlyHint":false,"destructiveHint":false,"idem /** The tool definition, minus `execute` (which is yours, below the marker). */ export const adoptPetTool = { name: "adopt-pet", - description: "Adopt a pet — this finalizes the adoption paperwork. Returns the pet.", + title: "Adopt Pet", + description: "Adopt a pet - this finalizes the adoption paperwork. Returns the pet.", inputSchema: adoptPetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeAdoptPet is the agent-facing one. */ +export async function fetchAdoptPet(input: AdoptPetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}/adopt`, { method: "POST", signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -59,7 +68,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { // // This tool changes things, so the user is always asked first. The // // confirmation lives in the generated region: it cannot be edited away. // const confirmed = await requestUserConfirmation( - // "Allow the agent to: Adopt a pet — this finalizes the adoption paperwork. Returns the pet.", + // "Allow the agent to: Adopt a pet - this finalizes the adoption paperwork. Returns the pet.", // ); // if (!confirmed) { // return { @@ -80,7 +89,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "adopt-pet". @@ -93,7 +102,7 @@ export async function registerAdoptPet(signal?: AbortSignal): Promise { * The user is asked to confirm every call (built into the generated region). */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeAdoptPet(input: AdoptPetInput) { diff --git a/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts index 6018ac3..b3c5f23 100644 --- a/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/create-pet.webmcp.ts @@ -1,6 +1,6 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Add a new pet to the store. Returns the pet. * @@ -36,14 +36,23 @@ export const createPetHints = {"readOnlyHint":false,"destructiveHint":false,"ide /** The tool definition, minus `execute` (which is yours, below the marker). */ export const createPetTool = { name: "create-pet", + title: "Create Pet", description: "Add a new pet to the store. Returns the pet.", inputSchema: createPetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeCreatePet is the agent-facing one. */ +export async function fetchCreatePet(input: CreatePetInput, signal?: AbortSignal) { + const data = await callApi("/pets", { method: "POST", body: { name: input.name, tag: input.tag }, signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -84,7 +93,7 @@ export async function registerCreatePet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "create-pet". @@ -97,7 +106,7 @@ export async function registerCreatePet(signal?: AbortSignal): Promise { * The user is asked to confirm every call (built into the generated region). */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeCreatePet(input: CreatePetInput) { diff --git a/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts index 57b556b..d2425a2 100644 --- a/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/delete-pet.webmcp.ts @@ -1,6 +1,6 @@ -import { toolDisabled } from "./runtime.webmcp"; +import { callApi, toolDisabled } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Remove a pet from the store permanently * @@ -15,7 +15,7 @@ export const deletePetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const deletePetHints = {"readOnlyHint":false,"destructiveHint":true,"idem /** The tool definition, minus `execute` (which is yours, below the marker). */ export const deletePetTool = { name: "delete-pet", + title: "Delete Pet", description: "Remove a pet from the store permanently", inputSchema: deletePetInputSchema, annotations: { readOnlyHint: false, untrustedContentHint: false, + consequentialHint: true, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeDeletePet is the agent-facing one. */ +export async function fetchDeletePet(input: DeletePetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}`, { method: "DELETE", signal }); + return data; +} + /** * Withheld: this tool is not registered, so agents cannot see or pick * it. The registration below stays commented until you enable the tool @@ -80,7 +89,7 @@ export async function registerDeletePet(signal?: AbortSignal): Promise { // ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "delete-pet". diff --git a/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts b/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts index 608f0e1..1238b39 100644 --- a/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/get-pet.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * Get one pet, including its owner's contact details. Returns the pet. * @@ -15,7 +15,7 @@ export const getPetInputSchema = { "properties": { "id": { "type": "string", - "description": "Id." + "description": "The unique identifier of the pet." } }, "required": [ @@ -32,14 +32,23 @@ export const getPetHints = {"readOnlyHint":true,"destructiveHint":false,"idempot /** The tool definition, minus `execute` (which is yours, below the marker). */ export const getPetTool = { name: "get-pet", + title: "Get Pet", description: "Get one pet, including its owner's contact details. Returns the pet.", inputSchema: getPetInputSchema, annotations: { readOnlyHint: true, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeGetPet is the agent-facing one. */ +export async function fetchGetPet(input: GetPetInput, signal?: AbortSignal) { + const data = await callApi(`/pets/${input.id}`, { method: "GET", signal }); + return data; +} + /** * Register this tool with WebMCP. Call it once on page load, or use * registerAllTools() from the generated index.ts. Skips quietly when the @@ -67,7 +76,7 @@ export async function registerGetPet(signal?: AbortSignal): Promise { ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "get-pet". @@ -77,7 +86,7 @@ export async function registerGetPet(signal?: AbortSignal): Promise { * whenever you like; the contract above never changes. */ // -// ⚠ webmcp-codegen flagged these response fields as likely PII: owner.email. +// ! webmcp-codegen flagged these response fields as likely PII: owner.email. // Everything you return reaches the agent. Leave those fields out of what you // return unless the agent genuinely needs them, and say so in a comment if you keep them. export async function executeGetPet(input: GetPetInput) { diff --git a/examples/openapi-petstore/src/webmcp/index.ts b/examples/openapi-petstore/src/webmcp/index.ts index 516b540..7f38ef6 100644 --- a/examples/openapi-petstore/src/webmcp/index.ts +++ b/examples/openapi-petstore/src/webmcp/index.ts @@ -33,4 +33,6 @@ export async function registerAllTools(signal?: AbortSignal): Promise { console.warn("[webmcp-codegen] a tool failed to register:", error); } } + // Drop journey definitions into ./journeys/ and re-run `generate`: + // the next barrel registers every createJourney() export it finds there. } diff --git a/examples/openapi-petstore/src/webmcp/journey.webmcp.ts b/examples/openapi-petstore/src/webmcp/journey.webmcp.ts new file mode 100644 index 0000000..a67ca24 --- /dev/null +++ b/examples/openapi-petstore/src/webmcp/journey.webmcp.ts @@ -0,0 +1,267 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours - journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT - one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS - ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema - the definition lives in one place - and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; + /** + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. + */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; +} + +/** + * A step with no backend call of its own - it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; +} + +export type JourneyStep = ToolStep | FreeStep; + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute - your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + +export function createJourney(def: JourneyDef) { + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + + async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), + description: stepDescription(step), + inputSchema: stepInput(step), + annotations: { readOnlyHint: stepReadOnly(step) }, + execute: async (input, context) => { + try { + await runStep(step, input as Json, context?.signal); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready - call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false, consequentialHint: true }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now - for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts b/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts index 470b9a9..9568cdf 100644 --- a/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/list-pets.webmcp.ts @@ -1,6 +1,6 @@ import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- /** * List all pets in the store. Returns an array of pets. * @@ -34,14 +34,23 @@ export const listPetsHints = {"readOnlyHint":true,"destructiveHint":false,"idemp /** The tool definition, minus `execute` (which is yours, below the marker). */ export const listPetsTool = { name: "list-pets", + title: "List Pets", description: "List all pets in the store. Returns an array of pets.", inputSchema: listPetsInputSchema, annotations: { readOnlyHint: true, untrustedContentHint: true, + consequentialHint: false, }, }; +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeListPets is the agent-facing one. */ +export async function fetchListPets(input: ListPetsInput, signal?: AbortSignal) { + const data = await callApi("/pets", { method: "GET", query: { status: input.status }, signal }); + return data; +} + /** * Register this tool with WebMCP. Call it once on page load, or use * registerAllTools() from the generated index.ts. Skips quietly when the @@ -69,7 +78,7 @@ export async function registerListPets(signal?: AbortSignal): Promise { ); } -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- /** * What actually happens when the agent calls "list-pets". diff --git a/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts b/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts index ae9fb54..01067b1 100644 --- a/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts +++ b/examples/openapi-petstore/src/webmcp/runtime.webmcp.ts @@ -13,10 +13,16 @@ export interface WebMcpToolResult { /** A tool as the browser runtime understands it. */ export interface WebMcpToolDefinition { name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; description: string; inputSchema?: Record; /** Hints the agent reads to decide how careful to be with this tool. */ - annotations?: { readOnlyHint?: boolean; untrustedContentHint?: boolean }; + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; execute: ( input: Record, context?: { signal?: AbortSignal }, @@ -27,7 +33,7 @@ export interface WebMcpToolDefinition { export interface ModelContext { registerTool( tool: WebMcpToolDefinition, - options?: { signal?: AbortSignal }, + options?: { signal?: AbortSignal; exposedTo?: string[] }, ): Promise; } @@ -52,6 +58,30 @@ export function getModelContext(): ModelContext | null { return modelContext ?? null; } +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + /** * Call your API from the page. Same origin by default (pass a full URL when * the API lives on another host), always with the signed-in user's session @@ -91,12 +121,27 @@ export async function callApi( } } -/** Wrap a result in the MCP shape, so tool bodies stay one line. */ +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + "\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; return { - content: [ - { type: "text", text: typeof data === "string" ? data : JSON.stringify(data, null, 2) }, - ], + content: [{ type: "text", text: fitted }], }; } diff --git a/packages/codegen/README.md b/packages/codegen/README.md index c25044c..3b147b3 100644 --- a/packages/codegen/README.md +++ b/packages/codegen/README.md @@ -1,5 +1,5 @@
- + webmcp-stack

@webmcp-stack/codegen

@@ -9,7 +9,7 @@ MIT license

- Docs | + Docs | GitHub | Issues

@@ -17,11 +17,15 @@ --- -Your spec already knows your tools. One command writes them, wires them into your app, and gets out of the way. Part of [webmcp-stack](https://github.com/SouravInsights/webmcp-stack), the open-source developer stack for [WebMCP](https://github.com/webmachinelearning/webmcp). +Hand-writing a WebMCP tool for every action is a chore, and the descriptions drift from the API they describe. If you already maintain an OpenAPI spec or validation schemas, you already have the source of truth. + +`@webmcp-stack/codegen` turns that source into WebMCP tools as real TypeScript files in your repo. It writes the safe defaults, withholds anything risky until you enable it, scaffolds a skill file so your own coding agent knows the rules, and ships a `verify` command you can gate in CI. + +Part of [webmcp-stack](https://github.com/SouravInsights/webmcp-stack), the open-source developer stack for [WebMCP](https://github.com/webmachinelearning/webmcp). ## Quick start -Zero install, zero config: +No install, no config for the first run: ```bash npx @webmcp-stack/codegen generate @@ -29,105 +33,106 @@ npx @webmcp-stack/codegen generate One run finds your OpenAPI spec (monorepos included), finds the package that is your web app, and: -- **generates working tools** in `src/webmcp/`: reads call your API out of the box, mutations are generated disabled (working code, one uncomment away) -- **filters what shouldn't be a tool**: webhooks skipped, auth and admin endpoints flagged and disabled -- **wires registration into your app** (two additive lines for Next.js and Vite, reported with undo instructions) +- writes one `.webmcp.ts` file per endpoint +- gives read tools working implementations that call your API with the signed-in user's session +- **withholds** writes and destructive tools: the working code is generated, but the tool is not registered until you enable it +- skips webhooks, flags auth and admin endpoints +- wires registration into your app with two additive lines +- scaffolds a skill file for your coding agent, and a journey factory if you want one -Preview first, write nothing: +Preview everything without writing: ```bash npx @webmcp-stack/codegen generate --dry-run ``` -Then start your app, open it in Chrome with `#enable-webmcp-testing`, and ask the agent to use one of your tools. - -## No OpenAPI spec? Use your schemas +Measure the surface before you ship, and gate it in CI: -Most React/Next.js apps don't have one. If your app validates with schemas (zod, valibot, arktype), you can declare agent-facing actions straight from them, and a schema entry can also *refine* an OpenAPI operation (your contract and words, the endpoint's mechanics): +```bash +npx @webmcp-stack/codegen verify +``` -```js -// codegen.config.mjs -import { defineConfig } from "@webmcp-stack/codegen"; -import { openapi, schema } from "@webmcp-stack/codegen/sources"; -import { tools } from "@webmcp-stack/codegen/outputs"; -import { CreateTripInput } from "./src/schemas"; +Browse and test the tools locally: -export default defineConfig({ - sources: [ - openapi({ spec: "./openapi.yaml" }), - schema({ - tools: [ - // operation fuses this with the spec's createTrip endpoint into one tool - { name: "create-trip", schema: CreateTripInput, operation: "createTrip" }, - ], - }), - ], - outputs: [tools({ outDir: "./src/webmcp" })], -}); +```bash +npx @webmcp-stack/codegen dev ``` -Every field gets text an agent can act on: your `.describe()` words stay verbatim, constraints are rendered as plain language ("A number from 30 to 600."), and anything still silent gets a marked draft the audit reports on. +Then open your app in Chrome with `chrome://flags/#enable-webmcp-testing` enabled, and let an agent use a tool. Other browsers work with the [WebMCP polyfill](https://github.com/webmachinelearning/webmcp-polyfill). -Working with a literal `` instead? The `form` output annotates it in place with WebMCP's declarative attributes (`{ name: "add-to-timesheet", schema: Entry, form: "./src/TimesheetForm.tsx" }` plus `form` in `outputs`), so the agent fills the visible form and a human keeps the final click on writes. +## What you get -## Safety is part of generation +**A small, intent-level surface.** Not every endpoint should be a tool. Reads register so you can see the surface; writes and destructive tools stay withheld until you turn them on one at a time. -This is not a dumb API → WebMCP converter. Giving agents access to application actions is a new security surface, so the generator analyzes what every endpoint actually is: read-only, write, destructive, auth-boundary, or sensitive/PII-related. Every tool gets a safety classification and WebMCP hints, the audit pass runs inside `generate` (errors block, exit codes for CI), and higher-risk tools are generated disabled so you explicitly decide what agents can touch. The goal is that you stay in control of the agent-facing surface instead of blindly exposing every endpoint. +**Handshake grouping.** When a begin/end pair (request an upload, then complete it) is really one action split across two calls, the generator merges them into a single withheld tool and threads the first response into the second by exact name. A pair it cannot thread is skipped with a note, never guessed. -## The dashboard +**Journeys.** For a goal that takes several calls and ends in one confirmed write, you write a small file in `src/webmcp/journeys/` and the generator scaffolds the rest: a shared draft, one tool per step, and a submit gate a human confirms. See [Journeys](https://webmcp.souravinsights.com/docs/journeys). -```bash -npx @webmcp-stack/codegen dev -``` +**A skill file for your coding agent.** Every run writes `.agents/skills/webmcp-tools/SKILL.md`, where Claude Code, AGENTS.md, and the generic standard already look. Your agent follows the same naming, description, and safety rules the generator uses. -A local control panel for your WebMCP surface, the way Scalar is for APIs or Storybook is for components: browse your tools, inspect and edit metadata, toggle tools on and off, and run any tool directly to check it works. Edits save to `.webmcp-codegen.json` and survive regeneration. Nothing is added to your app. +**Safety as part of generation.** Every endpoint is classified read, write, or destructive, corrected when the name disagrees. The audit runs inside `generate`: it names PII in outputs, descriptions that try to instruct the agent, and auth or admin endpoints. Errors block generation and set an exit code for CI; warnings report and continue. -## What a generated tool looks like +**Descriptions you can trust.** Constraints become sentences ("A number from 30 to 600."), every description says what the tool returns, and free-text outputs are marked as untrusted. Your own text is kept in full. Chrome's 500 and 150 character budgets are treated as guidance, so `verify` warns on an overrun instead of shortening your words to pass a counter. -One file per endpoint, like `delete-pet.webmcp.ts`: +**Regeneration that never clobbers your code.** The API contract lives above a marker line and regenerates freely; your `execute()` body lives below it and is never touched. Hand-edited generated regions produce a `.new` file to merge, never a silent overwrite. -```ts -// ─── webmcp-codegen: generated. Do not edit this region. ─── -export const deletePetInputSchema = { /* derived from your spec */ }; -export type DeletePetInput = { id: string }; -export async function registerDeletePet(signal?: AbortSignal) { - // Registers the tool; mutations ask the user to confirm, always. -} -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +## Where this fits + +This tool generates from a contract, not from a scan of your codebase. That is a deliberate choice, not a hidden limitation: + +- **It needs a source you maintain.** An OpenAPI spec, or the validation schemas your app already uses (zod, valibot, arktype, TypeBox). If neither exists yet, there is nothing good to generate from, and writing that source is the work to do first. +- **The source has to be good.** A tool's description is part of the prompt a model reasons over. A vague or stale contract produces vague or stale tools; what an agent can do is bounded by what your contract actually says. +- **One thing done well.** A generator that claims to work on any codebase, by guessing intent from source code, tends to work well nowhere. This one asks for a contract and rewards you for keeping it current. -export async function executeDeletePet(input: DeletePetInput) { - // This tool starts disabled: it changes things. To enable it, delete the - // line below and uncomment the code. - return toolDisabled("delete-pet.webmcp.ts"); +If you have an OpenAPI spec or maintained schemas, you are exactly who this is for. If you do not, the `schema` source lets you declare tools from schemas you write by hand in the meantime. + +## A generated tool + +One file per endpoint, like `create-trip.webmcp.ts`: + +```ts +// --- webmcp-codegen: generated. Do not edit this region. --- +export const createTripTool = { + name: "create-trip", + title: "Create Trip", + description: "Create a new trip. Returns the trip.", + inputSchema: createTripInputSchema, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, +}; - // const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); - // return toolResult(data); +// Journeys and your own code compose this raw caller; executeCreateTrip wraps it. +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips", { method: "POST", body: { ... }, signal }); + return data; } -``` -- **Real files in your repo**: readable, editable, no runtime dependency on this package -- **Working implementations**: path params, query strings, and JSON bodies built from the spec; session cookies included -- **Safety classification on every tool**: read/write/destructive, with WebMCP hints computed -- **An audit pass built into `generate`**: PII-in-response warnings, agent-instructing description linting, auth-boundary checks; errors block generation (exit codes for CI) -- **Regeneration never clobbers your code**: the contract regenerates above the marker, your code below it survives; hand-edited generated regions produce a `.new` file, never a silent overwrite +// Withheld, so the registration is generated but commented out, confirmation included: +// const confirmed = await requestUserConfirmation("Allow the agent to: Create a new trip..."); +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- +``` ## CLI | Command | What it does | |---|---| -| `webmcp-codegen generate` | Generate/update tools, wire registration (audit runs by default) | +| `webmcp-codegen generate` | Generate and update tools, wire registration (the audit runs by default) | | `generate --dry-run` | Preview everything, write nothing | | `generate --watch` | Re-generate when source files change | | `generate --force` | Write files even when the audit reports errors | -| `generate --suggest PATH` | Ask the LLM layer which schemas in PATH are worth declaring (proposals only) | | `generate --spec PATH` / `--out DIR` | Overrides without a config file | +| `webmcp-codegen verify` | Score the surface against the standard; exits 1 on errors | +| `verify --url URL` | Also check a deployed page serves an origin trial token | | `webmcp-codegen dev` | Open the tools dashboard (`--port N` to change the port) | | `webmcp-codegen init` | Write `codegen.config.mjs` for full control (needs the package installed) | ## Config -Structure lives in `codegen.config.mjs` (code); remembered choices and per-tool -overrides live in `.webmcp-codegen.json` (data, safe with npx, commit it). +Structure lives in `codegen.config.mjs` (code). Remembered choices and per-tool +overrides live in `.webmcp-codegen.json` (data, safe with npx, meant to be committed). ```js // codegen.config.mjs @@ -139,18 +144,34 @@ export default defineConfig({ sources: [openapi({ spec: "./openapi.yaml" })], outputs: [tools({ outDir: "./src/webmcp" })], safety: { - piiFields: ["internalId"], // extend the built-in PII heuristics - exclude: ["internal"], // skip tools by name or route substring + piiFields: ["internalId"], // extend the built-in PII heuristics + exclude: ["internal"], // skip tools by name or route substring }, - // llm: { apiKey: "..." }, // opt-in advisory layer: drafts and suggestions - // rendered as proposals, never applied, never blocking }); ``` +No OpenAPI spec? Declare tools from the schemas you already use, and a schema +entry can also refine an endpoint from your spec: + +```js +// import { schema } from "@webmcp-stack/codegen/sources"; +// import { CreateTripInput } from "./src/schemas"; + +sources: [ + openapi({ spec: "./openapi.yaml" }), + schema({ + tools: [{ name: "create-trip", schema: CreateTripInput, operation: "createTrip" }], + }), +], +``` + +Annotating a literal `` instead? The `form` output wires WebMCP's +declarative attributes onto it, so an agent fills the controls a person can see. + ## Requirements -- Node.js ≥ 20 -- To *use* the generated tools in a browser: Chrome 146+ with `#enable-webmcp-testing` (or the WebMCP polyfill) +- Node.js 20 or newer +- To *use* the generated tools in a browser: enable `chrome://flags/#enable-webmcp-testing` for local development (Chrome 149+, Edge 150+). Production pages join the WebMCP origin trial, or use the polyfill. ## License diff --git a/packages/codegen/assets/journey.webmcp.ts b/packages/codegen/assets/journey.webmcp.ts new file mode 100644 index 0000000..a67ca24 --- /dev/null +++ b/packages/codegen/assets/journey.webmcp.ts @@ -0,0 +1,267 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours - journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT - one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS - ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema - the definition lives in one place - and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; + /** + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. + */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; +} + +/** + * A step with no backend call of its own - it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; +} + +export type JourneyStep = ToolStep | FreeStep; + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute - your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + +export function createJourney(def: JourneyDef) { + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + + async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), + description: stepDescription(step), + inputSchema: stepInput(step), + annotations: { readOnlyHint: stepReadOnly(step) }, + execute: async (input, context) => { + try { + await runStep(step, input as Json, context?.signal); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready - call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false, consequentialHint: true }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now - for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/packages/codegen/assets/skill/SKILL.md b/packages/codegen/assets/skill/SKILL.md new file mode 100644 index 0000000..356ca92 --- /dev/null +++ b/packages/codegen/assets/skill/SKILL.md @@ -0,0 +1,130 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + + + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default - follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch - it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") - that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld - generated but not registered - until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` - the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick + provides: ["locationObject"], + }, + // Freeform step: no backend call - collects input straight into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2-5 steps. More means two journeys. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file - `verify` flags it. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/packages/codegen/evals/skill/README.md b/packages/codegen/evals/skill/README.md new file mode 100644 index 0000000..19df1d0 --- /dev/null +++ b/packages/codegen/evals/skill/README.md @@ -0,0 +1,75 @@ +# Skill-file evals + +The skill file (`assets/skill/SKILL.md`) is treated like code: a wording +change that makes agents write worse WebMCP tools should show up as a failed +test, not a vibe. This directory is that test rig, following the OpenAI +eval-skills writeup and philschmid's testing-skills post: +**prompt -> captured run -> checks -> score, over multiple trials.** + +## Layout + +- `fixture/` - beenthere-lite, a real generated surface (place search, trip + list, withheld trip creation) with the skill file installed at + `.agents/skills/webmcp-tools/`. Every case starts from a fresh copy. +- `cases.json` - the prompt set. `core` cases gate; `negative` cases prove + the skill doesn't leak into unrelated work; `control` cases are + informational only. +- `run.mjs` - the harness and its deterministic graders. +- `results/` - timestamped JSON reports (gitignored). + +## Running + +You need an agent CLI on PATH (Codex, Claude Code, ...). The command template +is `AGENT_CMD`, with `{PROMPT}` substituted and shell-quoted: + +```sh +AGENT_CMD='codex exec --json "{PROMPT}"' node run.mjs +AGENT_CMD='claude -p "{PROMPT}" --output-format json' node run.mjs --trials 5 +node run.mjs --case journey-document-trip # one case +node run.mjs --selftest # grade the graders, no agent +``` + +Exit code is 0 only when every gated case passes every trial. Control cases +never gate. + +## The sharpest case + +`journey-document-trip`: the prompt asks for a flow around trip creation on +a *retrospective* (been-there) product. The right answer composes the +generated tools into `createJourney(...)`, names it like `document-trip`, +and gates creation behind the human. `plan-trip` / `book-trip` match a +forbidden pattern - that naming failure passes every mechanical lint yet is +semantically wrong, which is exactly the class of mistake the skill file +exists to prevent. + +Its twin, `journey-without-skill`, runs the same prompt with the skill +directory deleted. It doesn't gate: it's the absorption detector. If it +starts passing reliably across models, the models internalized the practices +and the skill can retire. + +## Deterministic checks, LLM judgment only via the agent under test + +Graders look at files, not transcripts: expected files satisfying +include/exclude regexes, the generated region's marker preserved byte-for-byte, +description budgets measured, the tree unchanged on unrelated prompts. Adding +an LLM-as-judge pass is possible later - constrain it to a structured schema +(`overall_pass`, per-check results) so scores diff across runs - but the +fixtures were chosen so regex + structure carry the verdict. + +## The operating loop + +1. A real failure - from a user report, a docs change, a model regression - + becomes a case here first. +2. Tune the skill until the case is at ~100% pass rate across trials. +3. The case joins the regression set permanently. +4. Run the suite on skill edits and on new flagship models; run the control + case occasionally to detect absorption. + +Results stay local under `results/` by design: this harness needs an agent +CLI and model access, so it runs on demand, not in CI. + +## Refreshing the fixture + +The fixture is committed generated output (see `fixture/README.md`). When +the generator's templates change, regenerate it so the eval baseline is what +users actually receive. diff --git a/packages/codegen/evals/skill/cases.json b/packages/codegen/evals/skill/cases.json new file mode 100644 index 0000000..54b74ec --- /dev/null +++ b/packages/codegen/evals/skill/cases.json @@ -0,0 +1,123 @@ +{ + "version": 1, + "comment": "The skill-file eval set. Each case: a prompt, an agent run against a fresh copy of fixture/, deterministic checks on the result. See README.md.", + "agent_cmd_default": "codex exec --json \"{PROMPT}\"", + "trials_default": 3, + "cases": [ + { + "id": "trigger-new-read-tool", + "kind": "core", + "prompt": "Add a webmcp tool that lets agents look up the current weather for a city.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp", + "all": [ + "name: \"(get|search|check|lookup)-[a-z-]*weather[a-z-]*\"", + "readOnlyHint: true", + "registerTool\\(" + ], + "none": ["\\bfetch\\s*\\(\\s*[\"'`]https?://(?!/v1/)"] + } + ] + }, + { + "id": "journey-document-trip", + "kind": "core", + "comment": "The sharpest fixture: retrospective naming is right, plan/book is auto-fail.", + "prompt": "Look at the webmcp tools in this repo. Creating a trip needs a resolved locationObject from the place search, and the human must confirm before anything is created. Write the journey for this flow.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp/journeys", + "all": ["createJourney\\(", "submit", "\"(document|record|log|write|capture)-trip\""], + "none": ["\"(plan|book)-trip\"", "\\bfetch\\s*\\("] + } + ] + }, + { + "id": "enable-withheld-write", + "kind": "core", + "prompt": "Enable the create-trip tool so agents can use it.", + "checks": [ + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" }, + { + "type": "file-matches", + "path": "src/webmcp/create-trip.webmcp.ts", + "all": ["\\n await modelContext\\.registerTool\\(", "requestUserConfirmation"], + "none": [] + } + ] + }, + { + "id": "improve-description", + "kind": "core", + "comment": "Descriptions live in the generated region (do-not-edit) or the overrides file - the skill teaches overrides, so the marker must survive either way.", + "prompt": "The create-trip description doesn't tell agents what comes back. Improve it.", + "checks": [ + { + "type": "any-of", + "checks": [ + { + "type": "file-matches", + "path": ".webmcp-codegen.json", + "all": ["create-trip", "description"], + "none": [] + }, + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" } + ] + } + ] + }, + { + "id": "lazy-prompt-quality", + "kind": "core", + "prompt": "make my webmcp tools good", + "checks": [ + { + "type": "tool-description-budget", + "dir": "src/webmcp", + "max_tool": 500, + "max_param": 150 + }, + { "type": "marker-preserved", "path": "src/webmcp/create-trip.webmcp.ts" } + ] + }, + { + "id": "negative-typo", + "kind": "negative", + "prompt": "Fix the typo in README-APP.md", + "checks": [ + { "type": "file-matches", "path": "README-APP.md", "all": [], "none": ["\\bteh\\b"] }, + { "type": "no-webmcp-diff" } + ] + }, + { + "id": "negative-unit-test", + "kind": "negative", + "prompt": "Write a small unit test for a date-formatting helper (tests/format.test.ts).", + "checks": [{ "type": "no-webmcp-diff" }] + }, + { + "id": "negative-dark-mode", + "kind": "negative", + "prompt": "Add a dark-mode toggle to the settings page.", + "checks": [{ "type": "no-webmcp-diff" }] + }, + { + "id": "journey-without-skill", + "kind": "control", + "comment": "Same prompt as journey-document-trip with the skill REMOVED. Informational: if this starts passing reliably, models absorbed the practices and the skill can retire.", + "skill": false, + "prompt": "Look at the webmcp tools in this repo. Creating a trip needs a resolved locationObject from the place search, and the human must confirm before anything is created. Write the journey for this flow.", + "checks": [ + { + "type": "any-file-matches", + "dir": "src/webmcp/journeys", + "all": ["createJourney\\(", "submit", "\"(document|record|log|write|capture)-trip\""], + "none": ["\"(plan|book)-trip\"", "\\bfetch\\s*\\("] + } + ] + } + ] +} diff --git a/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md new file mode 100644 index 0000000..356ca92 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/.agents/skills/webmcp-tools/SKILL.md @@ -0,0 +1,130 @@ +--- +name: webmcp-tools +description: Build, improve, or review this repo's WebMCP tools and journeys - the *.webmcp.ts files that expose site capabilities to AI agents. Use when creating a new tool, editing a tool's name/description/schema, enabling a withheld tool, or defining a multi-step journey. +--- + + + +# WebMCP tools in this repo + +WebMCP tools run in the visitor's browser, with the signed-in user's session, +while the agent calling them may also be reading attacker-influenced page +content. Every tool you write is both an API and a security surface. The rules +below exist so an agent-facing tool is correct by default - follow them even +when the user's request is casual. + +## Naming + +- Verb-first, intent-shaped, max 30 characters: `list-trips`, + `add-bucket-list-destination`. Never method-first (`get-v1-trips`), never + numbered (`get-pricing-2`). +- Name the user's intent, not the endpoint. `POST /search` is a read named + `search-...`; `POST /orders/{id}/cancel` is destructive named `cancel-...`. +- **Understand the product before naming anything.** What is it, who uses it, + and when in the user's life does this action happen? If that is not written + down in the repo, ask the user before naming intent-level tools or journeys. + Wrong-tense or wrong-role names pass every mechanical check and are still + wrong: on a journal for trips you've *been on*, the flow is `document-trip`, + never `plan-trip`. This is the failure no linter can catch - it is your job. + +## Descriptions + +- Say what the tool does, when to use it, and what it returns: + "Create a new trip. Returns the trip." +- Max 500 characters for the tool, max 150 per parameter. Turn constraints + into sentences: "A number from 30 to 600." +- Never instruct the agent or encode flow control in a description + ("always call X first") - that is steering. Prerequisites belong in a + journey, not in prose. +- If a field's value can only come from another tool (a resolved place object, + a server id), say so in that field's description. + +## Safety and exposure + +- Reads are registered immediately. Writes and destructive tools stay + withheld - generated but not registered - until the user deliberately + enables one. Never enable a write tool without being asked. +- Mutating tools confirm each call with the human via + `requestUserConfirmation`. That call lives in the generated region; never + move or remove it. +- Free-text outputs get `untrustedContentHint: true` - the agent must not + treat user-written content as the site speaking. +- **The schema is not the security boundary.** `execute` must call the app's + real endpoint or action layer, so server-side validation runs on every + call. Never wire `execute` to return canned data or bypass the app's own + flow (cache invalidation, navigation, stores). + +## The execute contract + +- Never throw for failure. The browser maps a rejected `execute` to a bare + `UnknownError` and discards your message. Return `toolError(message)` / + `asToolError(error)` so the agent can read and recover. (Cancellation is + the one exception: let `AbortError` propagate.) +- Return via `toolResult(data)` and keep outputs under ~1.5K characters - + summarize or paginate rather than dumping. +- When a call changes what is on screen, make it visible: navigate, + invalidate a query, dispatch an event. The human is watching the page. + +## Editing generated files + +- Each `*.webmcp.ts` has a generated region between the + `webmcp-codegen` markers - never edit inside it; regeneration rewrites it. + Your work goes below the marker (the `execute` body) or in + `.webmcp-codegen.json` (description, enabled, and field-text overrides, which + survive regeneration and always win over generated text). +- After editing tools, run `npx @webmcp-stack/codegen verify` and fix what it + reports. + +## Journeys (multi-step flows) + +Reach for a journey when a goal takes several calls with shared state, when +an input can't be invented (a resolved place object), or when a spend should +be gated (an eligibility check before a paid generation). Journey files live +in the `journeys/` folder inside the generated tools directory +(`src/webmcp/journeys/document-trip.webmcp.ts`); re-run `verify` after +writing one. Pattern: + +```ts +// src/webmcp/journeys/document-trip.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { getAutocompleteTool, fetchGetAutocomplete } from "../get-autocomplete.webmcp"; +import { executeCreateTrip, type CreateTripInput } from "../create-trip.webmcp"; + +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + // Tool-backed step: inherits the generated tool's description and schema; + // you write only what lands in the draft. + "search-places": { + tool: getAutocompleteTool, + call: (input, signal) => fetchGetAutocomplete({ input: String(input.input) }, signal), + store: (places) => ({ locationObject: places }), // store the resolved pick + provides: ["locationObject"], + }, + // Freeform step: no backend call - collects input straight into the draft. + "set-details": { + description: "Set the trip's title and dates.", + input: { type: "object", properties: { title: { type: "string" } }, required: ["title"] }, + provides: ["title"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft as CreateTripInput, // assemble the real tool's input + run: executeCreateTrip, // the existing tool does the work + }, +}); +``` + +- 2-5 steps. More means two journeys. +- Tool-backed steps reuse the generated tool's contract and its raw caller + (`fetchX`); the submit's `run` is the real write tool's `execute`, so its + confirmation and validation still apply. Never write a direct `fetch` in a + journey file - `verify` flags it. +- Keep real writes in the submit gate. A tool-backed step inherits the composed + tool's read-only hint, and a free step that only stores input is read-only + too. Never route a mutating call through a step: it would be advertised as + safe and skip the confirmation the submit performs. diff --git a/packages/codegen/evals/skill/fixture/README-APP.md b/packages/codegen/evals/skill/fixture/README-APP.md new file mode 100644 index 0000000..c1ea7ac --- /dev/null +++ b/packages/codegen/evals/skill/fixture/README-APP.md @@ -0,0 +1,13 @@ +# beenthere-lite + +A journal for trips you've been on. Document the places, write the story, +share the memory. (Not a planner: users come here *after* the trip.) + +## Dev + +The stack is a small web app with an API at the same origin (`/v1/...`) and +WebMCP tools under `src/webmcp/`, generated from `spec.json` with +webmcp-codegen. Tools register on page load via `registerAllTools()` from +`src/webmcp/index.ts`. + +teh editor opens after a trip is created. diff --git a/packages/codegen/evals/skill/fixture/README.md b/packages/codegen/evals/skill/fixture/README.md new file mode 100644 index 0000000..d13cb69 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/README.md @@ -0,0 +1,8 @@ +# beenthere-lite (eval fixture) + +A memory-journal app for trips you've *been on* - deliberately NOT a travel +planner. Generated by `@webmcp-stack/codegen generate --spec spec.json` and +committed as the baseline every eval case starts from. + +Refresh after template changes: re-run the command above into a scratch dir +and copy the result over this folder. diff --git a/packages/codegen/evals/skill/fixture/spec.json b/packages/codegen/evals/skill/fixture/spec.json new file mode 100644 index 0000000..704608c --- /dev/null +++ b/packages/codegen/evals/skill/fixture/spec.json @@ -0,0 +1,113 @@ +{ + "openapi": "3.0.0", + "info": { "title": "beenthere-lite", "version": "1.0.0" }, + "paths": { + "/v1/places/autocomplete": { + "get": { + "operationId": "getAutocomplete", + "summary": "Search places by free text.", + "parameters": [ + { + "name": "input", + "in": "query", + "required": true, + "schema": { "type": "string", "description": "The place search text, e.g. Lisbon." } + } + ], + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "type": "object", + "properties": { + "placeId": { "type": "string" }, + "fullAddress": { "type": "string" }, + "types": { "type": "array", "items": { "type": "string" } }, + "country": { "type": "string" }, + "lat": { "type": "number" }, + "lng": { "type": "number" }, + "name": { "type": "string" } + }, + "required": ["placeId", "fullAddress", "types", "country", "lat", "lng", "name"] + } + } + } + } + } + } + } + }, + "/v1/trips/": { + "get": { + "operationId": "listTrips", + "summary": "List the signed-in user's trips.", + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "array", + "items": { + "type": "object", + "properties": { "id": { "type": "string" }, "title": { "type": "string" } } + } + } + } + } + } + } + }, + "post": { + "operationId": "createTrip", + "summary": "Create a new trip.", + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { + "title": { "type": "string", "description": "The trip's display title." }, + "startDate": { "type": "string", "format": "date" }, + "locationObject": { + "type": "object", + "description": "The resolved place from the autocomplete endpoint.", + "properties": { + "placeId": { "type": "string" }, + "fullAddress": { "type": "string" }, + "types": { "type": "array", "items": { "type": "string" } }, + "country": { "type": "string" }, + "lat": { "type": "number" }, + "lng": { "type": "number" }, + "name": { "type": "string" } + }, + "required": ["placeId", "fullAddress", "types", "country", "lat", "lng", "name"] + } + }, + "required": ["title", "locationObject"] + } + } + } + }, + "responses": { + "200": { + "description": "ok", + "content": { + "application/json": { + "schema": { + "type": "object", + "properties": { "id": { "type": "string" }, "title": { "type": "string" } } + } + } + } + } + } + } + } + } +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts new file mode 100644 index 0000000..8c10748 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/create-trip.webmcp.ts @@ -0,0 +1,167 @@ +import { callApi, toolDisabled } from "./runtime.webmcp"; + +// --- webmcp-codegen: generated. Do not edit this region. --- +/** + * Create a new trip. Returns the trip. + * + * Source: POST /v1/trips/ (openapi). Risk: write-confirm. + * Starts withheld: not registered until you enable it (see registerCreateTrip below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const createTripInputSchema = { + "type": "object", + "properties": { + "title": { + "type": "string", + "description": "The trip's display title." + }, + "startDate": { + "type": "string", + "format": "date", + "description": "Start date. A date (YYYY-MM-DD)." + }, + "locationObject": { + "type": "object", + "description": "The resolved place from the autocomplete endpoint.", + "properties": { + "placeId": { + "type": "string", + "description": "The unique identifier of the place." + }, + "fullAddress": { + "type": "string", + "description": "Full address." + }, + "types": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Types." + }, + "country": { + "type": "string", + "description": "Country." + }, + "lat": { + "type": "number", + "description": "Lat." + }, + "lng": { + "type": "number", + "description": "Lng." + }, + "name": { + "type": "string", + "description": "Name." + } + }, + "required": [ + "placeId", + "fullAddress", + "types", + "country", + "lat", + "lng", + "name" + ] + } + }, + "required": [ + "title", + "locationObject" + ] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type CreateTripInput = { "title": string; "startDate"?: string; "locationObject": { "placeId": string; "fullAddress": string; "types": string[]; "country": string; "lat": number; "lng": number; "name": string } }; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const createTripHints = {"readOnlyHint":false,"destructiveHint":false,"idempotentHint":false,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const createTripTool = { + name: "create-trip", + title: "Create Trip", + description: "Create a new trip. Returns the trip.", + inputSchema: createTripInputSchema, + annotations: { + readOnlyHint: false, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeCreateTrip is the agent-facing one. */ +export async function fetchCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "POST", body: { title: input.title, startDate: input.startDate, locationObject: input.locationObject }, signal }); + return data; +} + +/** + * Withheld: this tool is not registered, so agents cannot see or pick + * it. The registration below stays commented until you enable the tool + * (uncomment it and the body of executeCreateTrip, or flip it in the + * dashboard and regenerate). + */ +export async function registerCreateTrip(signal?: AbortSignal): Promise { + void signal; + // const modelContext = getModelContext(); + // if (!modelContext) return; + // await modelContext.registerTool( + // { + // ...createTripTool, + // execute: async (input, context) => { + // // Cancellation wins over everything, including the confirmation. + // context?.signal?.throwIfAborted(); + // // This tool changes things, so the user is always asked first. The + // // confirmation lives in the generated region: it cannot be edited away. + // const confirmed = await requestUserConfirmation( + // "Allow the agent to: Create a new trip. Returns the trip.", + // ); + // if (!confirmed) { + // return { + // content: [{ type: "text", text: "The user declined this action." }], + // isError: true, + // }; + // } + // // The browser has already validated the agent's input against the schema. + // // A failure returns a readable result; it never throws (see asToolError). + // try { + // return await executeCreateTrip(input as CreateTripInput, context?.signal); + // } catch (error) { + // return asToolError(error); + // } + // }, + // }, + // { signal }, + // ); +} + +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- + +/** + * What actually happens when the agent calls "create-trip". + * + * Default implementation: calls POST /v1/trips/ from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + * + * This tool is write-confirm: it changes things. + * The user is asked to confirm every call (built into the generated region). + */ +export async function executeCreateTrip(input: CreateTripInput, signal?: AbortSignal) { + // This tool is withheld: nothing registers it, so agents cannot see + // or call it. To enable it, uncomment the request below and the + // registration above, and add toolResult to the import. + void signal; // passed to fetch once you enable the call below + return toolDisabled("create-trip.webmcp.ts"); + + // const data = await fetchCreateTrip(input, signal); + // return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts new file mode 100644 index 0000000..94bac69 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/get-autocomplete.webmcp.ts @@ -0,0 +1,96 @@ +import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; + +// --- webmcp-codegen: generated. Do not edit this region. --- +/** + * Search places by free text. Returns an array of autocomplete. + * + * Source: GET /v1/places/autocomplete (openapi). Risk: safe-read. + * Starts enabled (see executeGetAutocomplete below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const getAutocompleteInputSchema = { + "type": "object", + "properties": { + "input": { + "type": "string", + "description": "The place search text, e.g. Lisbon." + } + }, + "required": [ + "input" + ] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type GetAutocompleteInput = { "input": string }; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const getAutocompleteHints = {"readOnlyHint":true,"destructiveHint":false,"idempotentHint":true,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const getAutocompleteTool = { + name: "get-autocomplete", + title: "Get Autocomplete", + description: "Search places by free text. Returns an array of autocomplete.", + inputSchema: getAutocompleteInputSchema, + annotations: { + readOnlyHint: true, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeGetAutocomplete is the agent-facing one. */ +export async function fetchGetAutocomplete(input: GetAutocompleteInput, signal?: AbortSignal) { + const data = await callApi("/v1/places/autocomplete", { method: "GET", query: { input: input.input }, signal }); + return data; +} + +/** + * Register this tool with WebMCP. Call it once on page load, or use + * registerAllTools() from the generated index.ts. Skips quietly when the + * browser has no WebMCP runtime. + * + * Pass an AbortSignal to unregister later: controller.abort(). + */ +export async function registerGetAutocomplete(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + ...getAutocompleteTool, + // The browser has already validated the agent's input against the schema. + // A failure returns a readable result; it never throws (see asToolError). + execute: async (input, context) => { + try { + return await executeGetAutocomplete(input as GetAutocompleteInput, context?.signal); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); +} + +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- + +/** + * What actually happens when the agent calls "get-autocomplete". + * + * Default implementation: calls GET /v1/places/autocomplete from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + */ +export async function executeGetAutocomplete(input: GetAutocompleteInput, signal?: AbortSignal) { + const data = await fetchGetAutocomplete(input, signal); + // Make the effect visible: an agent acts while a human watches this + // page. If this call changes what is on screen, update the UI here + // (navigate, invalidate a query, dispatch an event). + return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/index.ts b/packages/codegen/evals/skill/fixture/src/webmcp/index.ts new file mode 100644 index 0000000..5d6b5bb --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/index.ts @@ -0,0 +1,34 @@ +/** + * Generated by webmcp-codegen. This file is fully regenerated on every run. + * Import registerAllTools() once at app startup: + * + * import { registerAllTools } from "./webmcp"; + * await registerAllTools(); + */ + +import { registerGetAutocomplete } from "./get-autocomplete.webmcp"; +import { registerListTrips } from "./list-trips.webmcp"; +import { registerCreateTrip } from "./create-trip.webmcp"; + +const registrations = [ + registerGetAutocomplete, + registerListTrips, + registerCreateTrip +]; + +/** + * Register every generated tool with WebMCP. One tool failing (for example + * because the page's Permissions-Policy disables tools) never takes the + * others down with it. The failure is logged and registration continues. + */ +export async function registerAllTools(signal?: AbortSignal): Promise { + for (const register of registrations) { + try { + await register(signal); + } catch (error) { + console.warn("[webmcp-codegen] a tool failed to register:", error); + } + } + // Drop journey definitions into ./journeys/ and re-run `generate`: + // the next barrel registers every createJourney() export it finds there. +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts new file mode 100644 index 0000000..a67ca24 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/journey.webmcp.ts @@ -0,0 +1,267 @@ +/** + * Written by webmcp-codegen on every `generate` run. Do not edit by hand; + * your changes will be lost. This file is fully ours - journey definitions + * (your code) live in journeys/*.webmcp.ts and import createJourney from here. + * + * createJourney: multi-step agent flows with a shared draft and one submit. + * + * The three pieces, literally: + * + * 1. THE DRAFT - one plain object per page load (`let draft = {}` below). + * Nothing fancier: steps write their results into it, the submit reads + * from it. It dies with the page; a half-finished journey does not + * survive a reload, which is what you want. + * + * 2. STEP TOOLS - ordinary registered WebMCP tools, one per step, named + * "-" (e.g. "document-trip-search-places"). A step's + * execute stores what it produced into the draft, then replies with what + * is still missing, so the agent always knows the next move. + * + * 3. THE SUBMIT GATE - one more registered tool, "-submit". Its + * execute, in order: refuses with the list of missing steps, asks the + * human to confirm, runs the real tool's execute with the assembled + * input, clears the draft. There is no way to submit around it, because + * the real input only exists inside build(draft). + */ + +import { + asToolError, + getModelContext, + requestUserConfirmation, + toolError, + toolResult, + type WebMcpToolResult, +} from "./runtime.webmcp"; + +type Json = Record; + +/** + * A step backed by an existing generated tool. The step inherits the tool's + * description and input schema - the definition lives in one place - and + * calls the raw caller the generated file exports (fetchGetAutocomplete, + * not the agent-facing execute wrapper). You write only what's new: which + * slice of the result lands in the draft. + */ +export interface ToolStep { + /** The generated tool object, e.g. getAutocompleteTool. */ + tool: { + description?: string; + inputSchema?: Json; + /** The generated tool's annotations. The step inherits readOnlyHint from it. */ + annotations?: { readOnlyHint?: boolean }; + }; + /** + * The raw caller the generated file exports. Receives the step's input + * plus the draft so far, so a later step can feed on an earlier one's + * stored fields. + */ + call: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** Map the call's result into the draft fields this step leaves behind. */ + store: (result: unknown) => Json; + /** Draft fields this step leaves behind. Submit waits for all of them. */ + provides: string[]; + /** Override the agent-facing description. Default: the tool's own. */ + description?: string; + /** Override the agent-facing input schema. Default: the tool's own. */ + input?: Json; + /** + * Override the read-only hint. Default: the composed tool's own hint, so a + * step that composes a read stays read-only and a step that composes a + * write does not. Set this only when you know the step's behavior differs. + */ + readOnly?: boolean; +} + +/** + * A step with no backend call of its own - it collects input into the draft + * ("set the title and dates"). With a `run`, it can do work first; whatever + * `run` returns is stored. Without one, the input is stored verbatim. + */ +export interface FreeStep { + description: string; + input: Json; + provides: string[]; + run?: (input: Json, signal: AbortSignal | undefined, draft: Readonly) => Promise; + /** + * Override the read-only hint. Default: true when the step has no `run` + * (it only stores its input in the draft), false when `run` can do work. + */ + readOnly?: boolean; +} + +export type JourneyStep = ToolStep | FreeStep; + +export interface JourneyDef { + /** Journey name; step tools derive from it ("document-trip-search-places"). */ + name: string; + /** The one sentence every step repeats to the agent, so the goal survives. */ + goal: string; + steps: Record; + submit: { + /** What the human confirms, e.g. "Create the trip and open the editor." */ + description: string; + /** Assemble the real tool's input from the draft. This is your code. */ + build: (draft: Readonly) => unknown; + /** The existing tool's execute - your real endpoint runs here. */ + run: (input: never, signal?: AbortSignal) => Promise; + }; +} + +function isToolStep(step: JourneyStep): step is ToolStep { + return "tool" in step; +} + +/** Chrome's published budget for one tool description. */ +const TOOL_DESCRIPTION_MAX = 500; + +/** The ASCII cut marker; fitText reserves its length before slicing. */ +const ELLIPSIS = "..."; + +/** Fit machine-composed text to a budget, reserving room for the ellipsis. */ +function fitText(text: string, budget: number): string { + if (text.length <= budget) return text; + const slice = text.slice(0, Math.max(ELLIPSIS.length, budget - ELLIPSIS.length)); + const wordEnd = slice.lastIndexOf(" "); + return `${(wordEnd > 0 ? slice.slice(0, wordEnd) : slice).trimEnd()}${ELLIPSIS}`; +} + +/** + * Whether calling this step can change anything outside the draft. A step + * that composes a generated read tool inherits the tool's readOnlyHint; a + * free step that only stores input is a read; everything else defaults to + * "not read-only", because a step's `call`/`run` is user code we cannot + * inspect. We never advertise a write as safe just because it is a step. + */ +function stepReadOnly(step: JourneyStep): boolean { + if (step.readOnly !== undefined) return step.readOnly; + if (isToolStep(step)) return step.tool.annotations?.readOnlyHint === true; + return step.run === undefined; +} + +/** "document-trip-search-places" -> "Document Trip Search Places" (native UIs). */ +function toTitle(kebab: string): string { + return kebab + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + +export function createJourney(def: JourneyDef) { + /** The shared draft. Page-scoped on purpose: reloads start clean. */ + let draft: Json = {}; + + /** Every (step, field) pair the draft still lacks, as readable text. */ + function missing(): string[] { + return Object.entries(def.steps).flatMap(([key, step]) => + step.provides + .filter((field) => draft[field] === undefined) + .map((field) => `${def.name}-${key} (stores "${field}")`), + ); + } + + function stepDescription(step: JourneyStep): string { + const base = + step.description ?? (isToolStep(step) ? step.tool.description : undefined) ?? "Journey step."; + const suffix = ` Part of "${def.name}": ${def.goal}`; + if (base.length + suffix.length <= TOOL_DESCRIPTION_MAX) return base + suffix; + // The step's own sentence matters more than repeating the whole goal. + // Keep the journey name (the grouping signal), drop the goal, and fit the + // base so the composed text never exceeds the budget. + const tag = ` Part of "${def.name}".`; + return `${fitText(base, TOOL_DESCRIPTION_MAX - tag.length)}${tag}`; + } + + function stepInput(step: JourneyStep): Json { + if (step.input) return step.input; + if (isToolStep(step) && step.tool.inputSchema) return step.tool.inputSchema; + return { type: "object", properties: {} }; + } + + /** Run one step and store what it produced. */ + async function runStep(step: JourneyStep, input: Json, signal?: AbortSignal): Promise { + if (isToolStep(step)) { + const result = await step.call(input, signal, { ...draft }); + Object.assign(draft, step.store(result)); + return; + } + const stored = step.run ? await step.run(input, signal, { ...draft }) : input; + Object.assign(draft, stored); + } + + async function registerSteps(signal?: AbortSignal): Promise { + // Resolve the context here, not at createJourney() time: a browser or + // polyfill that installs WebMCP after this module loads must still get the + // journey registered. Generated tools look it up the same way. + const modelContext = getModelContext(); + if (!modelContext) return; + for (const [key, step] of Object.entries(def.steps)) { + await modelContext.registerTool( + { + name: `${def.name}-${key}`, + title: toTitle(`${def.name}-${key}`), + description: stepDescription(step), + inputSchema: stepInput(step), + annotations: { readOnlyHint: stepReadOnly(step) }, + execute: async (input, context) => { + try { + await runStep(step, input as Json, context?.signal); + const left = missing(); + return toolResult( + left.length === 0 + ? `Stored. The journey is ready - call ${def.name}-submit.` + : `Stored. Still needed: ${left.join(", ")}.`, + ); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + } + + async function registerSubmit(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + name: `${def.name}-submit`, + title: toTitle(`${def.name}-submit`), + description: def.submit.description, + inputSchema: { type: "object", properties: {} }, + annotations: { readOnlyHint: false, consequentialHint: true }, + execute: async (_input, context) => { + context?.signal?.throwIfAborted(); + const left = missing(); + if (left.length > 0) { + return toolError(`Not ready to submit. Call these first: ${left.join(", ")}.`); + } + const confirmed = await requestUserConfirmation( + `Allow the agent to: ${def.submit.description}`, + ); + if (!confirmed) return toolError("The user declined this action."); + try { + const result = await def.submit.run(def.submit.build(draft) as never, context?.signal); + draft = {}; // a submitted journey starts clean + return result as WebMcpToolResult; + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); + } + + return { + /** Call once on page load, next to registerAllTools(). */ + async register(signal?: AbortSignal): Promise { + await registerSteps(signal); + await registerSubmit(signal); + }, + /** What's in the draft right now - for the dashboard and for tests. */ + inspectDraft: (): Json => ({ ...draft }), + }; +} diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts new file mode 100644 index 0000000..07c5ea5 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/list-trips.webmcp.ts @@ -0,0 +1,89 @@ +import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp"; + +// --- webmcp-codegen: generated. Do not edit this region. --- +/** + * List the signed-in user's trips. Returns an array of trips. + * + * Source: GET /v1/trips/ (openapi). Risk: safe-read. + * Starts enabled (see executeListTrips below). + * Regenerate with: npx @webmcp-stack/codegen generate + */ + +/** The exact contract advertised to the agent. Derived from the API spec. Do not hand-edit. */ +export const listTripsInputSchema = { + "type": "object", + "properties": {}, + "required": [] +}; + +/** What `execute` receives. The browser validates agent input against the schema above. */ +export type ListTripsInput = Record; + +/** Safety hints computed by webmcp-codegen. Informational metadata for hosts and UIs. */ +export const listTripsHints = {"readOnlyHint":true,"destructiveHint":false,"idempotentHint":true,"untrustedContentHint":true} as const; + +/** The tool definition, minus `execute` (which is yours, below the marker). */ +export const listTripsTool = { + name: "list-trips", + title: "List Trips", + description: "List the signed-in user's trips. Returns an array of trips.", + inputSchema: listTripsInputSchema, + annotations: { + readOnlyHint: true, + untrustedContentHint: true, + consequentialHint: false, + }, +}; + +/** The bare request, without the agent-facing result wrapping. Journeys + * and your own code compose this; executeListTrips is the agent-facing one. */ +export async function fetchListTrips(input: ListTripsInput, signal?: AbortSignal) { + const data = await callApi("/v1/trips/", { method: "GET", signal }); + return data; +} + +/** + * Register this tool with WebMCP. Call it once on page load, or use + * registerAllTools() from the generated index.ts. Skips quietly when the + * browser has no WebMCP runtime. + * + * Pass an AbortSignal to unregister later: controller.abort(). + */ +export async function registerListTrips(signal?: AbortSignal): Promise { + const modelContext = getModelContext(); + if (!modelContext) return; + await modelContext.registerTool( + { + ...listTripsTool, + // The browser has already validated the agent's input against the schema. + // A failure returns a readable result; it never throws (see asToolError). + execute: async (input, context) => { + try { + return await executeListTrips(input as ListTripsInput, context?.signal); + } catch (error) { + return asToolError(error); + } + }, + }, + { signal }, + ); +} + +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- + +/** + * What actually happens when the agent calls "list-trips". + * + * Default implementation: calls GET /v1/trips/ from this page, with the + * signed-in user's session. Replace it with your app's own API client + * whenever you like; the contract above never changes. + * + * Calls the API on this page's own origin (same-origin by default). + */ +export async function executeListTrips(input: ListTripsInput, signal?: AbortSignal) { + const data = await fetchListTrips(input, signal); + // Make the effect visible: an agent acts while a human watches this + // page. If this call changes what is on screen, update the UI here + // (navigate, invalidate a query, dispatch an event). + return toolResult(data); +} \ No newline at end of file diff --git a/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts new file mode 100644 index 0000000..01067b1 --- /dev/null +++ b/packages/codegen/evals/skill/fixture/src/webmcp/runtime.webmcp.ts @@ -0,0 +1,190 @@ +/** + * Generated by webmcp-codegen. This file is fully regenerated on every run. + * Do not edit by hand; your changes will be lost. + */ + +/** The result shape tools return (same as MCP tool results). */ +export interface WebMcpToolResult { + content: { type: "text"; text: string }[]; + isError?: boolean; + [key: string]: unknown; +} + +/** A tool as the browser runtime understands it. */ +export interface WebMcpToolDefinition { + name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; + description: string; + inputSchema?: Record; + /** Hints the agent reads to decide how careful to be with this tool. */ + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; + execute: ( + input: Record, + context?: { signal?: AbortSignal }, + ) => unknown | Promise; +} + +/** The slice of the WebMCP draft spec the generated code uses. */ +export interface ModelContext { + registerTool( + tool: WebMcpToolDefinition, + options?: { signal?: AbortSignal; exposedTo?: string[] }, + ): Promise; +} + +/* Most browsers do not have WebMCP yet, so a missing model context is a + * normal page load, not an error. It gets one quiet log line per load, + * never one per tool. */ +let announcedUnavailable = false; + +/** + * The page's WebMCP model context, or null when the browser has none. + * Registration callers skip quietly on null; a missing runtime must never + * surface in the human-facing page (no throws, no console spam). + */ +export function getModelContext(): ModelContext | null { + const modelContext = (document as unknown as { modelContext?: ModelContext }).modelContext; + if (!modelContext && !announcedUnavailable) { + announcedUnavailable = true; + console.info( + "[webmcp-codegen] WebMCP is not available in this browser; tools were not registered.", + ); + } + return modelContext ?? null; +} + +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + +/** + * Call your API from the page. Same origin by default (pass a full URL when + * the API lives on another host), always with the signed-in user's session + * cookies. Throws on HTTP errors; returns the parsed JSON body, or raw text + * when the response is not JSON. Pass the signal from execute's context so + * a cancelled tool call stops the request. + */ +export async function callApi( + path: string, + options: { + method?: string; + query?: Record; + body?: unknown; + signal?: AbortSignal; + } = {}, +): Promise { + const url = new URL(path, window.location.origin); + for (const [key, value] of Object.entries(options.query ?? {})) { + if (value !== undefined && value !== null) url.searchParams.set(key, String(value)); + } + const response = await fetch(url, { + method: options.method ?? "GET", + credentials: "include", + headers: options.body !== undefined ? { "content-type": "application/json" } : undefined, + body: options.body !== undefined ? JSON.stringify(options.body) : undefined, + signal: options.signal, + }); + if (!response.ok) { + throw new Error("Request failed: " + response.status + " " + response.statusText); + } + if (response.status === 204) return null; + const text = await response.text(); + try { + return JSON.parse(text); + } catch { + return text; + } +} + +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + "\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ +export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; + return { + content: [{ type: "text", text: fitted }], + }; +} + +/** + * A failure the agent can read and act on. The one hard rule of the execute + * contract: never throw for failure. The browser maps a rejected execute to + * a bare UnknownError and discards the message, so a thrown failure teaches + * the agent nothing. Cancellation is the only exception, which is why + * asToolError re-throws AbortError. + */ +export function toolError(message: string): WebMcpToolResult { + return { content: [{ type: "text", text: message }], isError: true }; +} + +/** Convert any thrown failure into a readable error result. */ +export function asToolError(error: unknown): WebMcpToolResult { + if (error instanceof DOMException && error.name === "AbortError") throw error; + return toolError(error instanceof Error ? error.message : "The tool failed."); +} + +/** + * What a disabled tool tells the agent. The tool stays visible (so the agent + * knows it exists and can ask the human to enable it) but does nothing. + */ +export function toolDisabled(fileName: string): WebMcpToolResult { + return { + content: [ + { + type: "text", + text: + "This tool is currently disabled by the app developer. Ask them to enable it " + + "(uncomment the implementation in " + fileName + ").", + }, + ], + isError: true, + }; +} + +/** + * Default "agent proposes, human confirms" gate for write/destructive tools. + * Deliberately minimal (window.confirm). Replace it with your app's own + * dialog when you outgrow it. The point is that the user always gets a say. + */ +export function requestUserConfirmation(message: string): Promise { + return Promise.resolve(window.confirm(message)); +} diff --git a/packages/codegen/evals/skill/results/.gitignore b/packages/codegen/evals/skill/results/.gitignore new file mode 100644 index 0000000..224cbaf --- /dev/null +++ b/packages/codegen/evals/skill/results/.gitignore @@ -0,0 +1,3 @@ +# Eval reports stay local +* +!.gitignore diff --git a/packages/codegen/evals/skill/run.mjs b/packages/codegen/evals/skill/run.mjs new file mode 100644 index 0000000..29f0409 --- /dev/null +++ b/packages/codegen/evals/skill/run.mjs @@ -0,0 +1,395 @@ +#!/usr/bin/env node +/** + * The skill-file eval runner: prompt -> captured run -> checks -> score. + * + * Each case gets a fresh copy of fixture/ (plus the skill file, unless the + * case removes it), an agent runs the case's prompt inside it headlessly, + * and deterministic checks grade what the agent left behind - files, not + * transcripts, wherever possible. Behavior is nondeterministic: run trials + * and report pass RATES, never a single run's verdict. + * + * Usage: + * node run.mjs all cases, default trials + * node run.mjs --trials 5 more trials per case + * node run.mjs --case journey-document-trip + * node run.mjs --selftest grade the graders (no agent needed) + * + * The agent command comes from AGENT_CMD, defaulting to cases.json's + * `agent_cmd_default`; `{PROMPT}` is replaced by the shell-quoted prompt. + * Examples: + * AGENT_CMD='codex exec --json "{PROMPT}"' + * AGENT_CMD='claude -p "{PROMPT}" --output-format json' + */ + +import { spawn } from "node:child_process"; +import { cp, mkdir, mkdtemp, readdir, readFile, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join, relative } from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = dirname(fileURLToPath(import.meta.url)); +const FIXTURE = join(here, "fixture"); +const GENERATED_END = + "// --- webmcp-codegen: end generated. Your code below survives regeneration. ---"; + +// ---------------------------------------------------------------- graders +// Every grader: async (ctx) => null on pass, or a sentence on failure. +// ctx = { workdir, fixtureDir, transcript, check } + +/** Read file list under a dir, recursively, relative paths sorted. */ +async function tree(dir, base = dir) { + let out = []; + let entries = []; + try { + entries = await readdir(dir, { withFileTypes: true }); + } catch { + return out; + } + for (const entry of entries) { + const full = join(dir, entry.name); + if (entry.isDirectory()) out = out.concat(await tree(full, base)); + else out.push(relative(base, full)); + } + return out.sort(); +} + +async function readIn(dir, rel) { + try { + return await readFile(join(dir, rel), "utf8"); + } catch { + return null; + } +} + +const graders = { + async "file-exists"({ workdir, check }) { + return (await readIn(workdir, check.path)) === null ? `missing file: ${check.path}` : null; + }, + + async "file-matches"({ workdir, check }) { + const contents = await readIn(workdir, check.path); + if (contents === null) return `missing file: ${check.path}`; + return matchFailures(contents, check); + }, + + async "any-file-matches"({ workdir, check }) { + const files = (await tree(join(workdir, check.dir), join(workdir, check.dir))).filter((name) => + name.endsWith(".webmcp.ts"), + ); + for (const name of files) { + const contents = await readIn(join(workdir, check.dir), name); + if (contents !== null && matchFailures(contents, check) === null) return null; + } + return `no file under ${check.dir}/ satisfied all patterns (checked ${files.length} file${files.length === 1 ? "" : "s"})`; + }, + + /** The generated region (start of file through the end marker) must be byte-identical to the fixture's. */ + async "marker-preserved"({ workdir, fixtureDir, check }) { + const before = await readIn(fixtureDir, check.path); + const after = await readIn(workdir, check.path); + if (after === null) return `missing file: ${check.path}`; + if (before === null) return `fixture has no ${check.path} to compare against`; + const headOf = (text) => { + const index = text.indexOf(GENERATED_END); + return index === -1 ? text : text.slice(0, index + GENERATED_END.length); + }; + return headOf(after) === headOf(before) + ? null + : `${check.path}: the generated region was edited - regeneration will clobber it; descriptions and names move through .webmcp-codegen.json`; + }, + + /** Nothing under src/webmcp may differ from the fixture (negative cases). */ + async "no-webmcp-diff"({ workdir, fixtureDir }) { + const subdir = "src/webmcp"; + const beforeFiles = await tree(join(fixtureDir, subdir)); + const afterFiles = await tree(join(workdir, subdir)); + const changed = new Set(); + for (const name of new Set([...beforeFiles, ...afterFiles])) { + const before = await readIn(join(fixtureDir, subdir), name); + const after = await readIn(join(workdir, subdir), name); + if (before !== after) changed.add(name); + } + return changed.size === 0 + ? null + : `webmcp files changed on an unrelated prompt: ${[...changed].join(", ")}`; + }, + + /** Tool and parameter descriptions inside webmcp files stay within budget. */ + async "tool-description-budget"({ workdir, check }) { + const files = (await tree(join(workdir, check.dir), join(workdir, check.dir))).filter((name) => + name.endsWith(".webmcp.ts"), + ); + const offenders = []; + for (const name of files) { + const contents = await readIn(join(workdir, check.dir), name); + if (contents === null) continue; + const toolMatch = /description: "((?:[^"\\]|\\.)*)"/.exec(contents); + if (toolMatch && (toolMatch[1] ?? "").length > check.max_tool) { + offenders.push(`${name}: tool description ${(toolMatch[1] ?? "").length} chars`); + } + for (const match of contents.matchAll( + /(?:properties|input):[\s\S]*?description: "((?:[^"\\]|\\.)*)"/g, + )) { + if ((match[1] ?? "").length > check.max_param) { + offenders.push(`${name}: a parameter description runs ${(match[1] ?? "").length} chars`); + } + } + } + return offenders.length === 0 ? null : offenders.join("; "); + }, + + async "transcript-matches"({ transcript, check }) { + return new RegExp(check.pattern, "i").test(transcript) + ? null + : `transcript never matched /${check.pattern}/`; + }, + + async "any-of"(ctx) { + const failures = []; + for (const inner of ctx.check.checks) { + const failure = await graders[inner.type]({ ...ctx, check: inner }); + if (failure === null) return null; + failures.push(failure); + } + return `none of the alternatives passed: ${failures.join(" | ")}`; + }, +}; + +function matchFailures(contents, check) { + for (const pattern of check.all ?? []) { + if (!new RegExp(pattern, "m").test(contents)) return `missing expected pattern: /${pattern}/`; + } + for (const pattern of check.none ?? []) { + if (new RegExp(pattern, "m").test(contents)) return `matched forbidden pattern: /${pattern}/`; + } + return null; +} + +// ------------------------------------------------------------------- runs + +function shellQuote(text) { + return `'${text.replaceAll("'", "'\\''")}'`; +} + +function runAgent(command, prompt, workdir, timeoutMs) { + const cmd = command.replace("{PROMPT}", shellQuote(prompt)); + return new Promise((resolveRun) => { + const child = spawn(cmd, { cwd: workdir, shell: true, env: process.env }); + let out = ""; + child.stdout.on("data", (chunk) => (out += chunk)); + child.stderr.on("data", (chunk) => (out += chunk)); + const timer = setTimeout(() => child.kill("SIGTERM"), timeoutMs); + child.on("close", (code) => { + clearTimeout(timer); + resolveRun({ code, transcript: out }); + }); + }); +} + +async function runCase(caseDef, command, trials, report) { + const results = []; + for (let trial = 0; trial < trials; trial++) { + const workdir = await mkdtemp(join(tmpdir(), `webmcp-eval-${caseDef.id}-`)); + try { + await cp(FIXTURE, workdir, { recursive: true }); + if (caseDef.skill === false) + await rm(join(workdir, ".agents"), { recursive: true, force: true }); + const { transcript } = await runAgent( + command, + caseDef.prompt, + workdir, + Number(process.env.EVAL_TIMEOUT_MS ?? 10 * 60 * 1000), + ); + const failures = []; + for (const check of caseDef.checks) { + const failure = await graders[check.type]({ + workdir, + fixtureDir: FIXTURE, + transcript, + check, + }); + if (failure !== null) failures.push(`[${check.type}] ${failure}`); + } + results.push({ trial, pass: failures.length === 0, failures }); + } finally { + await rm(workdir, { recursive: true, force: true }); + } + } + const passed = results.filter((r) => r.pass).length; + const gate = caseDef.kind === "control" ? "informational" : passed === results.length; + report.cases.push({ + id: caseDef.id, + kind: caseDef.kind, + passed, + trials: results.length, + gate, + results, + }); + console.log( + ` ${gate === true ? "ok" : gate === "informational" ? "i" : "x"} ${caseDef.id}: ${passed}/${results.length} passed${caseDef.kind === "control" ? " (control)" : ""}`, + ); + for (const result of results.filter((r) => !r.pass)) { + for (const line of result.failures) console.log(` trial ${result.trial + 1}: ${line}`); + } +} + +// -------------------------------------------------------------- self-test + +async function selftest() { + const workdir = await mkdtemp(join(tmpdir(), "webmcp-eval-selftest-")); + try { + await cp(FIXTURE, workdir, { recursive: true }); + + const expectOutcome = async (label, check, expectFail, patch) => { + const dir = await mkdtemp(join(tmpdir(), "webmcp-eval-selftest-case-")); + await cp(FIXTURE, dir, { recursive: true }); + if (patch) await patch(dir); + const failure = await graders[check.type]({ + workdir: dir, + fixtureDir: FIXTURE, + transcript: "", + check, + }); + const gotFail = failure !== null; + console.log( + ` ${gotFail === expectFail ? "ok" : "x"} ${label}${gotFail && expectFail ? ` (${failure})` : ""}`, + ); + await rm(dir, { recursive: true, force: true }); + return gotFail === expectFail; + }; + + const outcomes = []; + const record = async (...args) => outcomes.push(await expectOutcome(...args)); + + await record( + "marker-preserved: untouched file passes", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + false, + ); + await record( + "marker-preserved: edited head fails", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + true, + async (dir) => { + const path = join(dir, "src/webmcp/create-trip.webmcp.ts"); + await writeFile( + path, + (await readFile(path, "utf8")).replace("Create a new trip.", "HAND EDITED text."), + ); + }, + ); + await record( + "marker-preserved: edit below the marker passes", + { type: "marker-preserved", path: "src/webmcp/create-trip.webmcp.ts" }, + false, + async (dir) => { + const path = join(dir, "src/webmcp/create-trip.webmcp.ts"); + await writeFile(path, `${await readFile(path, "utf8")}\n// user code below the marker\n`); + }, + ); + await record("no-webmcp-diff: untouched passes", { type: "no-webmcp-diff" }, false); + await record( + "no-webmcp-diff: a new tool fails", + { type: "no-webmcp-diff" }, + true, + async (dir) => writeFile(join(dir, "src/webmcp/new-tool.webmcp.ts"), "export {};\n"), + ); + await record( + "any-file-matches: a pattern the factory satisfies passes", + { + type: "any-file-matches", + dir: "src/webmcp", + all: ["export function createJourney"], + none: [], + }, + false, + ); + await record( + "any-file-matches: an empty journeys dir fails", + { type: "any-file-matches", dir: "src/webmcp/journeys", all: ["createJourney"], none: [] }, + true, + ); + await record( + "any-file-matches: forbidden pattern trips", + { type: "any-file-matches", dir: "src/webmcp", all: ["createTrip"], none: ["placeId"] }, + true, + ); + await record( + "any-file-matches: clean file passes", + { type: "any-file-matches", dir: "src/webmcp", all: ["createTrip"], none: [] }, + false, + ); + await record( + "tool-description-budget: fixture within budget passes", + { type: "tool-description-budget", dir: "src/webmcp", max_tool: 500, max_param: 150 }, + false, + ); + await record( + "tool-description-budget: over-budget description fails", + { type: "tool-description-budget", dir: "src/webmcp", max_tool: 500, max_param: 150 }, + true, + async (dir) => { + const path = join(dir, "src/webmcp/list-trips.webmcp.ts"); + await writeFile( + path, + (await readFile(path, "utf8")).replace( + /description: "((?:[^"\\]|\\.)*)"/, + `description: "Wordy. ${"Detail upon detail. ".repeat(40)}"`, + ), + ); + }, + ); + await record( + "transcript-matches: hits and misses", + { type: "transcript-matches", pattern: "no-such-phrase-in-empty-transcript" }, + true, + ); + + const ok = outcomes.every(Boolean); + await rm(workdir, { recursive: true, force: true }); + console.log(ok ? "\n Self-test passed." : "\n Self-test FAILED."); + return ok ? 0 : 1; + } catch (error) { + console.error(error); + return 1; + } +} + +// -------------------------------------------------------------------- main + +async function main() { + const args = process.argv.slice(2); + if (args.includes("--selftest")) process.exit(await selftest()); + + const cases = JSON.parse(await readFile(join(here, "cases.json"), "utf8")); + const command = process.env.AGENT_CMD ?? cases.agent_cmd_default; + const trials = Number(args[args.indexOf("--trials") + 1]) || cases.trials_default || 3; + const only = args.includes("--case") ? args[args.indexOf("--case") + 1] : undefined; + + const selected = cases.cases.filter((c) => !only || c.id === only); + if (selected.length === 0) { + console.error(`No case matching ${only}. Known: ${cases.cases.map((c) => c.id).join(", ")}`); + process.exit(2); + } + + console.log(`Running ${selected.length} case(s), ${trials} trial(s) each, via: ${command}\n`); + const report = { at: new Date().toISOString(), command, trials, cases: [] }; + for (const caseDef of selected) await runCase(caseDef, command, trials, report); + + const outDir = join(here, "results"); + await mkdir(outDir, { recursive: true }); + const outPath = join(outDir, `${report.at.replaceAll(":", "-")}.json`); + await writeFile(outPath, JSON.stringify(report, null, 2)); + + const gated = report.cases.filter((c) => c.kind !== "control"); + const allPassed = gated.every((c) => c.gate === true); + console.log( + `\n ${gated.filter((c) => c.gate === true).length}/${gated.length} gated cases fully passing - report: ${relative(process.cwd(), outPath)}`, + ); + process.exit(allPassed ? 0 : 1); +} + +main().catch((error) => { + console.error(error); + process.exit(1); +}); diff --git a/packages/codegen/package.json b/packages/codegen/package.json index 3615af4..feb6063 100644 --- a/packages/codegen/package.json +++ b/packages/codegen/package.json @@ -29,7 +29,8 @@ } }, "files": [ - "dist" + "dist", + "assets" ], "scripts": { "build": "tsup src/index.ts src/cli.ts src/sources/index.ts src/outputs/index.ts src/dev/server.ts src/dev/ui.ts --format esm --dts --sourcemap --clean", diff --git a/packages/codegen/src/cli-output.ts b/packages/codegen/src/cli-output.ts index 1169998..f15fbdb 100644 --- a/packages/codegen/src/cli-output.ts +++ b/packages/codegen/src/cli-output.ts @@ -31,7 +31,7 @@ function termWidth(): number { } /** - * The banner, printed once at the start of every command — a banner leads, + * The banner, printed once at the start of every command - a banner leads, * it never sits in the middle of a report. The full wordmark in cfonts * "tiny": solid block letters, small enough that webmcp-stack fits in 51 * columns. The color is the site's accent (#58a6ff), on a real terminal @@ -153,7 +153,7 @@ function wrapNames(names: string[], indent: string): string[] { return lines; } -/** The default summary output — written for humans, not machines. */ +/** The default summary output - written for humans, not machines. */ export function renderSummary( result: GenerateResult, setup: Setup, @@ -171,7 +171,7 @@ export function renderSummary( console.log(""); // What happened - console.log(` ${c.green("✓")} ${bold(`${tools.length} tools generated`)}`); + console.log(` ${c.green("ok")} ${bold(`${tools.length} tools generated`)}`); const parts = [`${enabled} ready to use`]; if (withheld > 0) parts.push(`${withheld} withheld until you enable them`); if (gated > 0) parts.push(`${gated} visible but disabled`); @@ -182,15 +182,15 @@ export function renderSummary( console.log(""); // Blocking errors first, one line each, in red. They stop the write, and a - // red wall that says exactly what to fix beats a green ✓ that doesn't mean it. + // red wall that says exactly what to fix beats a green ok that doesn't mean it. const errorFindings = findings.filter((f) => f.level === "error"); if (errorFindings.length > 0) { console.log( - ` ${c.red("✖")} ${bold(`${errorFindings.length} error${errorFindings.length === 1 ? "" : "s"}, nothing written`)}`, + ` ${c.red("x")} ${bold(`${errorFindings.length} error${errorFindings.length === 1 ? "" : "s"}, nothing written`)}`, ); for (const f of errorFindings) { const where = f.tool ? dim(` (${f.tool})`) : ""; - console.log(` ${c.red("✖")} ${f.message}${where}`); + console.log(` ${c.red("x")} ${f.message}${where}`); } console.log(""); } @@ -206,7 +206,7 @@ export function renderSummary( for (const group of groups) { const heading = group.heading.length > maxHeading - ? `${group.heading.slice(0, maxHeading - 1)}…` + ? `${group.heading.slice(0, maxHeading - 1)}...` : group.heading; console.log(dim(` ${heading}`)); } @@ -216,9 +216,9 @@ export function renderSummary( // Where things went const outDir = setup.config.outputs[0]?.outDir ?? "src/webmcp"; - console.log(` ${c.cyan("→")} ${bold("Files")} ${outDir}`); + console.log(` ${c.cyan("->")} ${bold("Files")} ${outDir}`); if (wiring && !wiring.alreadyWired) { - console.log(` ${c.cyan("→")} ${bold("Registration")} wired into your app`); + console.log(` ${c.cyan("->")} ${bold("Registration")} wired into your app`); } // Per-file notes (kept attributes, added names, unmatched controls). Capped // in the summary; --verbose lists them all. @@ -227,31 +227,27 @@ export function renderSummary( console.log(dim(` ${note}`)); } if (fileNotes.length > 6) { - console.log(dim(` …and ${fileNotes.length - 6} more (run with --verbose)`)); + console.log(dim(` ...and ${fileNotes.length - 6} more (run with --verbose)`)); } - console.log(""); - // LLM proposals: visually distinct from findings (◦, cyan), because a - // suggestion is not a fact. Nothing here was applied to anything. - if (result.suggestions.length > 0) { - console.log( - ` ${c.cyan("◦")} ${bold(`${result.suggestions.length} LLM suggestion${result.suggestions.length === 1 ? "" : "s"}`)} ${dim("(proposals only; nothing applied)")}`, - ); - for (const suggestion of result.suggestions) { - console.log(dim(` ◦ ${suggestion.message}`)); - } - console.log(""); + // Pipeline proposals (groupings, renames): the run's "look at this" lines. + for (const note of result.notes.slice(0, 6)) { + console.log(` ${c.cyan("-")} ${note}`); + } + if (result.notes.length > 6) { + console.log(dim(` ...and ${result.notes.length - 6} more (run with --verbose)`)); } + console.log(""); // Next step console.log(` ${bold("Next:")} ${c.cyan("npx @webmcp-stack/codegen dev")}`); console.log(dim(" Review your tools, edit descriptions, test them live")); console.log(""); - console.log(dim(` Docs: https://webmcp-stack.vercel.app/docs`)); + console.log(dim(` Docs: https://webmcp.souravinsights.com/docs`)); console.log(""); } -/** Verbose output — every tool, for when you want the full list. */ +/** Verbose output - every tool, for when you want the full list. */ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string): void { const { tools, findings, skipped } = result; @@ -304,14 +300,14 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string for (const tool of group) { const description = tool.description || "(no description)"; table.push([ - tool.name.length > nameWidth ? `${tool.name.slice(0, nameWidth - 1)}…` : tool.name, + tool.name.length > nameWidth ? `${tool.name.slice(0, nameWidth - 1)}...` : tool.name, tool.enabledByDefault ? c.green("enabled") : tool.withheld ? dim("withheld") : dim("disabled"), description.length > descWidth - 2 - ? `${description.slice(0, descWidth - 3)}…` + ? `${description.slice(0, descWidth - 3)}...` : description, ]); } @@ -320,7 +316,7 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } // Safety notes: grouped by what the person should do, with the affected - // tools listed compactly under each. 60 identical ⚠ lines teach nothing; + // tools listed compactly under each. 60 identical ! lines teach nothing; // 5 headed groups do. const errorFindings = findings.filter((f) => f.level === "error"); const groups = groupFindings(findings); @@ -329,11 +325,11 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string console.log(""); for (const f of errorFindings) { const where = f.tool ? dim(` (${f.tool})`) : ""; - console.log(` ${c.red("✖")} ${f.message}${where}`); + console.log(` ${c.red("x")} ${f.message}${where}`); } if (errorFindings.length > 0) console.log(""); for (const group of groups) { - console.log(` ${c.yellow("⚠")} ${group.heading}`); + console.log(` ${c.yellow("!")} ${group.heading}`); for (const line of wrapNames(group.items, " ")) { console.log(dim(line)); } @@ -341,15 +337,6 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } } - // LLM proposals, visually distinct from findings. - if (result.suggestions.length > 0) { - console.log(bold("LLM suggestions (proposals only; nothing applied):")); - for (const suggestion of result.suggestions) { - console.log(` ${c.cyan("◦")} ${dim(suggestion.message)}`); - } - console.log(""); - } - const notes = result.files.flatMap((file) => file.notes ?? []); if (notes.length > 0) { console.log(bold("File notes:")); @@ -360,6 +347,6 @@ export function renderVerbose(result: GenerateResult, setup: Setup, _cwd: string } console.log(dim(`Files: ${setup.config.outputs[0]?.outDir ?? "src/webmcp"}`)); - console.log(dim(`Docs: https://webmcp-stack.vercel.app/docs`)); + console.log(dim(`Docs: https://webmcp.souravinsights.com/docs`)); console.log(""); } diff --git a/packages/codegen/src/cli.ts b/packages/codegen/src/cli.ts index d4c41f6..3028471 100644 --- a/packages/codegen/src/cli.ts +++ b/packages/codegen/src/cli.ts @@ -19,19 +19,18 @@ */ import { existsSync } from "node:fs"; -import { writeFile } from "node:fs/promises"; +import { readdir, readFile, writeFile } from "node:fs/promises"; import { join, resolve } from "node:path"; -import { pathToFileURL } from "node:url"; import { parseArgs } from "node:util"; import * as clack from "@clack/prompts"; import { dim, printBanner, renderSummary } from "./cli-output.js"; -import { CONFIG_FILE_NAMES, loadConfig } from "./config.js"; +import { CONFIG_FILE_NAMES } from "./config.js"; import { loadDataFile, saveDataFile } from "./data-file.js"; import { findSpecs } from "./detect.js"; -import { findSchemaLibraries, findSchemaModules, findWebApps } from "./detect-app.js"; +import { findSchemaLibraries, findWebApps } from "./detect-app.js"; // Node prints an ExperimentalWarning the first time it type-strips a .ts -// module (which --suggest does to load user schemas). That warning is Node +// module (loading a user's codegen.config.ts or schema modules). That warning is Node // talking about its own internals, not something the developer can act on, // so it never belongs in our output. const realEmitWarning = process.emitWarning.bind(process); @@ -43,17 +42,20 @@ process.emitWarning = ((warning: unknown, ...rest: unknown[]) => { }) as typeof process.emitWarning; import { startDevServer } from "./dev/server.js"; -import { hostedLlmProvider, resolveLlmProvider, runLlmLayer } from "./llm.js"; import { debug, enableVerbose, error, info, success, warn } from "./logger.js"; import { runGenerate } from "./pipeline.js"; import { resolveSetup } from "./setup.js"; -import { schemaExportsToJson } from "./sources/schema.js"; -import type { CodegenConfig } from "./types.js"; -import { verifyTools, verifyUrl } from "./verify.js"; +import { + countJourneyTools, + type JourneyFileInput, + verifyJourneyFiles, + verifyTools, + verifyUrl, +} from "./verify.js"; import { applyWiring, planWiring, type WirePlan } from "./wire.js"; const HELP = ` -webmcp-codegen — generate WebMCP tools from your OpenAPI spec +webmcp-codegen - generate WebMCP tools from your OpenAPI spec Usage npx @webmcp-stack/codegen [command] [flags] @@ -71,8 +73,6 @@ Flags --verbose Show every tool, not just the summary --force Write files even when the audit reports errors --skip-audit Skip the safety report - --suggest Ask the LLM layer which of your schemas are worth declaring - --llm Improve the names and descriptions of the tools being generated (LLM) --url URL With verify: also check the deployed page is live for visitors --config PATH Use a config file at PATH --port N Dashboard port (default: 4700) @@ -85,7 +85,7 @@ Examples npx @webmcp-stack/codegen generate --verbose # see all 72 tools listed Docs - https://webmcp-stack.vercel.app/docs + https://webmcp.souravinsights.com/docs `; export interface CliFlags { @@ -98,10 +98,6 @@ export interface CliFlags { spec?: string; out?: string; port?: number; - /** `generate --llm`: polish the generated tools' descriptions and names. */ - llm?: boolean; - /** `generate --suggest`: LLM proposals for undeclared schemas. */ - suggest?: boolean; /** `verify --url `: also check the page is live for visitors. */ url?: string; help: boolean; @@ -121,8 +117,6 @@ async function main(): Promise { spec: { type: "string" }, out: { type: "string" }, port: { type: "string" }, - suggest: { type: "boolean" }, - llm: { type: "boolean" }, url: { type: "string" }, help: { type: "boolean", default: false }, }, @@ -138,8 +132,6 @@ async function main(): Promise { spec: values.spec, out: values.out, port: values.port ? Number.parseInt(values.port, 10) : undefined, - suggest: values.suggest, - llm: values.llm, url: values.url, help: values.help, }; @@ -161,7 +153,7 @@ async function main(): Promise { case "verify": return verify(flags); case "generate": - return flags.suggest ? suggest() : generate(flags); + return generate(flags); default: error(`Unknown command: ${command}`); info(HELP); @@ -224,7 +216,7 @@ async function init(): Promise { info("schemas; see the commented block in the config."); } info("\nEdit it to add sources, change the output directory, or set safety options."); - info("Docs: https://webmcp-stack.vercel.app/docs/configuration\n"); + info("Docs: https://webmcp.souravinsights.com/docs/configuration\n"); return 0; } @@ -318,165 +310,10 @@ async function dev(port: number): Promise { return 0; } -/** - * Resolve the LLM provider for an explicit opt-in (--llm / --suggest). With a - * configured key this returns it silently; without one, an interactive - * terminal offers the hosted tier / own key / skip. Non-interactive runs - * (CI) get undefined, because a prompt can never block a pipeline. Both LLM - * flags share this so the chooser is identical everywhere. - */ -async function resolveProviderForOptIn( - llm: CodegenConfig["llm"], -): Promise> { - const configured = resolveLlmProvider(llm ?? {}); - if (configured) return configured; - if (!process.stdout.isTTY) { - info( - "\n◦ The LLM layer is off: no provider configured and no WEBMCP_LLM_API_KEY / " + - "OPENAI_API_KEY in the environment. Nothing proposed.\n", - ); - return undefined; - } - const choice = await clack.select({ - message: "The LLM layer needs an API key. How do you want to proceed?", - options: [ - { - value: "hosted", - label: "Use the free hosted tier", - hint: "webmcp-stack's shared key, rate-limited", - }, - { - value: "own", - label: "Enter my own API key", - hint: "OpenRouter, OpenAI, or any OpenAI-compatible provider", - }, - { value: "skip", label: "Skip", hint: "run without LLM features" }, - ], - }); - if (clack.isCancel(choice) || choice === "skip") { - info("\n◦ Skipped. Nothing proposed.\n"); - return undefined; - } - if (choice === "hosted") return hostedLlmProvider(); - const key = await clack.password({ - message: "Paste your API key (input is hidden):", - validate: (value) => - !value || value.trim().length === 0 ? "The key cannot be empty." : undefined, - }); - if (clack.isCancel(key)) { - info("\n◦ Skipped. Nothing proposed.\n"); - return undefined; - } - const provider = resolveLlmProvider({ ...llm, apiKey: key.trim() }); - if (!provider) { - warn("\nThat key did not resolve to a provider. Nothing proposed.\n"); - return undefined; - } - info(dim(" Key used for this run only. To save it: export WEBMCP_LLM_API_KEY=...")); - return provider; -} - -/** - * `generate --suggest`: the LLM layer's tool-worthiness proposals. The tool - * finds the schema modules itself — nobody should have to pass a file path - * to get proposals. A proposal surface only: it reads the discovered schemas, - * asks, and prints. Nothing is declared, generated, or written; declaring is - * the developer's edit. - */ -async function suggest(): Promise { - printBanner(); - const cwd = process.cwd(); - - // The llm settings live in the config when there is one. A missing config is - // fine here: --suggest is itself the explicit opt-in, so env keys are enough. - let llm: CodegenConfig["llm"]; - try { - llm = (await loadConfig(cwd)).config.llm; - } catch { - llm = undefined; - } - - const modules = await findSchemaModules(cwd); - if (modules.length === 0) { - info( - "\n◦ No schema modules found. The tool looks for files named like " + - '"schemas.ts" or "models.ts" under packages/, src/, apps/, or lib/. ' + - "If yours live elsewhere, declare them directly in codegen.config.mjs.\n", - ); - return 0; - } - const provider = await resolveProviderForOptIn(llm); - if (!provider) return 0; - - const allSchemas: { name: string; schemaText: string }[] = []; - for (const modulePath of modules) { - let moduleExports: Record; - try { - moduleExports = await importSchemaModule(cwd, modulePath); - } catch (error) { - debug(`could not load ${modulePath}: ${error instanceof Error ? error.message : error}`); - continue; - } - const { schemas, skipped } = schemaExportsToJson(moduleExports, cwd); - for (const entry of skipped) { - debug(`skipped ${entry.name}: ${entry.reason}`); - } - allSchemas.push(...schemas); - } - if (allSchemas.length === 0) { - info("\n◦ Found schema modules but no loadable schemas in them. Nothing to propose on.\n"); - return 0; - } - - const spinner = clack.spinner(); - spinner.start("Asking the LLM which schemas are worth declaring"); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: llm ?? {} }, - { tools: [], findings: [], suggestExports: allSchemas }, - ); - spinner.stop("Done"); - - info(""); - if (suggestions.length === 0) { - info(" ◦ The provider had no proposals. Declare schemas by hand, as usual."); - } - for (const suggestion of suggestions) { - info(` ◦ ${suggestion.message}`); - } - info( - dim("\n Proposals only; nothing was written. Declare what you want in codegen.config.mjs.\n"), - ); - return 0; -} - -/** - * Load a user module for --suggest. TypeScript loads only via Node's native - * type stripping (22.18+ / 23.6+): the leaning choice from the spec, kept as - * the single code path so the CLI stays zero-dependency. The tradeoff lives - * here on purpose: older runtimes and extensionless barrel imports get a - * clear, actionable error instead of a second loader. - */ -async function importSchemaModule( - cwd: string, - modulePath: string, -): Promise> { - const absolute = resolve(cwd, modulePath); - try { - return (await import(pathToFileURL(absolute).href)) as Record; - } catch (error) { - throw new Error( - `Could not load "${modulePath}". If it is TypeScript, run on Node 22.18+ (or 23.6+) ` + - "and import the schema file directly (explicit .ts extension; extensionless barrel " + - "re-exports need a bundler), or point --suggest at a plain-JS module.\n" + - `Underlying error: ${error instanceof Error ? error.message : String(error)}`, - ); - } -} - /** * The tool standard, measured locally: runs the pipeline exactly as generate * would (nothing written), then reports how the registered surface holds up - * — names, descriptions, field text, annotations, surface size. Exits 1 on + * - names, descriptions, field text, annotations, surface size. Exits 1 on * error-level findings so CI can gate on it. */ async function verify(flags: CliFlags): Promise { @@ -502,19 +339,44 @@ async function verify(flags: CliFlags): Promise { }); const registered = result.tools.filter((tool) => !tool.withheld); - const checks = verifyTools(result.tools); + + // Journey files are the user's code, so verify can't get them from the + // pipeline's tool list - it reads the journeys/ folder of each tools + // output itself and lints what it finds there. Read them first: the surface + // count needs to include the tools they register at runtime. + const journeyInputs: JourneyFileInput[] = []; + for (const output of setup.config.outputs) { + if (output.kind !== "tools") continue; + const journeysDir = resolve(cwd, (output as { outDir?: string }).outDir ?? "", "journeys"); + let entries: string[] = []; + try { + entries = (await readdir(journeysDir)).filter((entry) => entry.endsWith(".webmcp.ts")); + } catch { + continue; + } + for (const entry of entries) { + journeyInputs.push({ + path: join("journeys", entry), + contents: await readFile(join(journeysDir, entry), "utf8"), + }); + } + } + + const journeyToolCount = countJourneyTools(journeyInputs); + const checks = verifyTools(result.tools, { journeyToolCount }); + checks.push(...verifyJourneyFiles(journeyInputs)); info(""); info(` ${setup.label}: ${result.tools.length} tools, ${registered.length} registered`); info(""); for (const check of checks) { - const mark = check.level === "ok" ? "✓" : check.level === "error" ? "✖" : "!"; + const mark = check.level === "ok" ? "ok" : check.level === "error" ? "x" : "!"; info(` ${mark} ${check.area}: ${check.summary}`); for (const finding of check.findings.slice(0, 8)) { info(` ${finding}`); } if (check.findings.length > 8) { - info(dim(` …and ${check.findings.length - 8} more`)); + info(dim(` ...and ${check.findings.length - 8} more`)); } } @@ -526,7 +388,7 @@ async function verify(flags: CliFlags): Promise { info(` ! ${finding.message}`); } if (pageFindings.length === 0) { - info(" ✓ Page checks passed"); + info(" ok Page checks passed"); } } @@ -563,7 +425,7 @@ async function generate(flags: CliFlags): Promise { const progress = flags.verbose ? (msg: string) => debug(msg) : undefined; // The data file carries the developer's dashboard edits and the last run's - // names. Both must reach the pipeline here — the dashboard is where edits + // names. Both must reach the pipeline here - the dashboard is where edits // are made, but this command is where files are written, and an edit that // only one of them reads does not survive. const data = await loadDataFile(cwd); @@ -579,7 +441,7 @@ async function generate(flags: CliFlags): Promise { }); if (!flags.verbose && result.tools.length > 0) { - info(` → Read ${result.tools.length + result.skipped.length} operations`); + info(` -> Read ${result.tools.length + result.skipped.length} operations`); } // Audit errors block the write. Instead of requiring --force on a re-run, @@ -633,31 +495,6 @@ async function generate(flags: CliFlags): Promise { }); } - // The LLM layer, on explicit opt-in only: polish the tools this run - // generated. Proposals print as `◦` lines; nothing is auto-applied, exit - // codes never change, and a failing provider is a note, not a failure. - if (flags.llm) { - const provider = await resolveProviderForOptIn(setup.config.llm); - if (provider) { - const spinner = clack.spinner(); - spinner.start("Improving names and descriptions"); - const suggestions = await runLlmLayer( - setup.config, - { tools: result.tools, findings: result.findings }, - undefined, - provider, - ); - spinner.stop("Done"); - if (suggestions.length === 0) { - info(" ◦ The provider had no proposals."); - } - for (const suggestion of suggestions) { - info(` ◦ ${suggestion.message}`); - } - info(""); - } - } - if (!flags.verbose) { renderSummary(result, setup, cwd, wiring); } diff --git a/packages/codegen/src/data-file.ts b/packages/codegen/src/data-file.ts index fb40ef1..4bd758e 100644 --- a/packages/codegen/src/data-file.ts +++ b/packages/codegen/src/data-file.ts @@ -1,7 +1,7 @@ /** * The remembered-choices file: `.webmcp-codegen.json` at the project root. * - * It is plain data — never code — so it works in the pure-npx flow (no + * It is plain data - never code - so it works in the pure-npx flow (no * install needed) and can be read and written safely by the CLI and the dev * dashboard alike. It holds two kinds of things: * diff --git a/packages/codegen/src/describe.test.ts b/packages/codegen/src/describe.test.ts index 70b44f4..d6aa95a 100644 --- a/packages/codegen/src/describe.test.ts +++ b/packages/codegen/src/describe.test.ts @@ -4,9 +4,42 @@ import { describeCandidateTool, describeConstraints, describeField, + FIELD_DESCRIPTION_MAX, + fitBudget, + TOOL_DESCRIPTION_MAX, } from "./describe.js"; import type { CandidateTool } from "./types.js"; +describe("fitBudget", () => { + it("passes text that fits through untouched", () => { + expect(fitBudget("Short text.", 150)).toBe("Short text."); + }); + + it("cuts overflow at a sentence boundary when one keeps most of the text", () => { + const text = `${"First sentence that goes on for a while. ".repeat(3)}Second sentence trails off at the very end with no stop`; + const fitted = fitBudget(text, 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith(".")).toBe(true); + expect(fitted).not.toContain("Second sentence"); + }); + + it("hard-cuts at a word boundary with an ellipsis when no good sentence break exists", () => { + const text = `${"a b c d e f g h i j k l m n o p q r s t u v w x y z ".repeat(6)}end`; + const fitted = fitBudget(text, 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith("...")).toBe(true); + expect(fitted).not.toContain("end"); + }); + + it("never exceeds the budget for a single unbroken token", () => { + // The regression that started this: a URL or token with no spaces used to + // come back over budget because the cut marker was appended after the cut. + const fitted = fitBudget("x".repeat(300), 150); + expect(fitted.length).toBeLessThanOrEqual(150); + expect(fitted.endsWith("...")).toBe(true); + }); +}); + describe("describeConstraints", () => { it("renders a number range the way the WebMCP docs do", () => { expect(describeConstraints({ type: "number", minimum: 30, maximum: 600 })).toBe( @@ -405,3 +438,35 @@ describe("describeCandidateInputs through nullable wrappers", () => { expect(candidate.synthesizedFields).toEqual([]); }); }); + +describe("description budgets", () => { + it("keeps long author field text verbatim instead of silently cutting it", () => { + const authored = `The notes field of the record. ${"More detail about things. ".repeat(9)}`; + const result = describeField("notes", { type: "string", description: authored.trim() }); + expect(result.synthesized).toBe(false); + // Chrome's budget is guidance, not a browser rule: the author's sentence + // survives and verify warns, rather than this layer dropping half of it. + expect(result.description.length).toBeGreaterThan(FIELD_DESCRIPTION_MAX); + expect(result.description).toContain("More detail about things."); + }); + + it("fits machine-drafted field text to the 150-character budget", () => { + // No author text, so the draft is ours to trim; the long pattern makes it + // overflow the budget. + const result = describeField("value", { type: "string", pattern: "a".repeat(200) }); + expect(result.synthesized).toBe(true); + expect(result.description.length).toBeLessThanOrEqual(FIELD_DESCRIPTION_MAX); + }); + + it("keeps long author tool text verbatim instead of cutting at 500", () => { + const candidate = { + name: "list-trips", + description: `List the trips. ${"A long explanation of everything this endpoint could ever do. ".repeat(12)}`, + inputSchema: { type: "object", properties: {} }, + outputSchema: { type: "array" }, + } as unknown as CandidateTool; + describeCandidateTool(candidate); + expect(candidate.description.length).toBeGreaterThan(TOOL_DESCRIPTION_MAX); + expect(candidate.description.startsWith("List the trips.")).toBe(true); + }); +}); diff --git a/packages/codegen/src/describe.ts b/packages/codegen/src/describe.ts index 5b4996f..9fbfd4f 100644 --- a/packages/codegen/src/describe.ts +++ b/packages/codegen/src/describe.ts @@ -13,6 +13,13 @@ * - Append, never replace. Author text (a spec description, a `.describe()`) * stays verbatim; synthesized constraints follow it. The author's words are * always the better text. + * - Bound the machine's words, never the author's. Chrome's 500/150 budgets + * are authoring guidance, not spec rules (the browser only rejects an + * empty description or a name outside 1-128 chars). So machine-drafted + * text is composed to fit the budget by construction, and author text is + * never silently shortened. If author text runs long, verify warns and + * the developer decides; losing half a sentence to a character counter is + * worse than a description that is a few characters over. * - Only fill silence. A field with no text at all gets a draft built from * its name, type, and constraints, and that field is marked as * machine-written so the audit can see it. Machine text is a floor, not a @@ -21,13 +28,47 @@ * audit is only meaningful if a CI run is reproducible. * * This module covers layers 1-3 of the assembly order (source text, merge, - * synthesis). Layer 4 (LLM drafts) is advisory and lives in llm.ts; layer 5 - * (overrides) lives in the pipeline's override step, applied last so it wins. + * synthesis). Overrides live in the pipeline's override step, applied last + * so they always win. */ import pluralize from "pluralize"; import type { CandidateTool, JsonSchema } from "./types.js"; +/** Chrome's published description budgets: 500 per tool, 150 per parameter. */ +export const TOOL_DESCRIPTION_MAX = 500; +export const FIELD_DESCRIPTION_MAX = 150; + +/** The ASCII cut marker; fit functions reserve its length before slicing. */ +const ELLIPSIS = "..."; + +/** + * Fit machine-drafted text to a character budget. A text that fits passes + * through untouched. One that overflows is cut at the last sentence boundary + * that keeps at least half the budget (a cut near the end keeps the thought, + * at the price of trailing sentences); otherwise it hard-cuts at a word + * boundary and ends with an ellipsis. The ellipsis is reserved inside the + * budget, so the result never exceeds it, even for a single unbroken token. + * + * Only call this on text this module generated. Author text is left alone + * on purpose (see the module comment). + */ +export function fitBudget(text: string, budget: number): string { + if (text.length <= budget) return text; + // Reserve room for the cut marker, so the result can never come back over + // budget when there is no space to cut at. + const slice = text.slice(0, budget - ELLIPSIS.length); + const sentenceEnd = Math.max( + slice.lastIndexOf(". "), + slice.lastIndexOf("! "), + slice.lastIndexOf("? "), + ); + if (sentenceEnd >= Math.floor(budget / 2)) return slice.slice(0, sentenceEnd + 1); + const wordEnd = slice.lastIndexOf(" "); + const body = wordEnd > 0 ? slice.slice(0, wordEnd) : slice; + return `${body.trimEnd()}${ELLIPSIS}`; +} + /** * Render a schema's constraints as plain-language sentences, or "" when there * is nothing worth saying. The wording mirrors the examples in Chrome's WebMCP @@ -154,7 +195,7 @@ function alreadyStatesConstraints(text: string, schema: JsonSchema): boolean { return values.length > 0 && values.every((value) => text.includes(value)); } -/** "purchaseDate" / "purchase_date" / "purchase-date" → "Purchase date". +/** "purchaseDate" / "purchase_date" / "purchase-date" -> "Purchase date". * Sentence case, matching the field text in Chrome's WebMCP examples; these * are machine drafts that the audit flags, not final copy. */ function humanizeFieldName(name: string): string { @@ -192,7 +233,7 @@ function patternSentence(name: string, schema: JsonSchema, noun?: string): strin if (last === "url" && subject) return `The URL of the ${subject}.`; if (last === "at" && words.length > 1) { // The stem's last word is the event ("captured"); anything before it is - // what it happened to ("email verified at" → the email). + // what it happened to ("email verified at" -> the email). const happenedTo = words.slice(0, -2).join(" ") || noun; if (happenedTo) return `When the ${happenedTo} was ${words[words.length - 2]}.`; } @@ -205,7 +246,7 @@ function patternSentence(name: string, schema: JsonSchema, noun?: string): strin } /** - * The noun a tool acts on, from its name: "create-trip" → "trip". Our own + * The noun a tool acts on, from its name: "create-trip" -> "trip". Our own * naming rules put the verb first, so the next segment that means something * is the noun. Only a fallback subject for pattern sentences. */ @@ -256,34 +297,50 @@ export function describeField( const authorText = raw && !isStubDescription(raw) ? raw : ""; const constraints = describeConstraints(schema); + let result: { description: string; synthesized: boolean }; + if (authorText) { const needsConstraints = constraints && !alreadyStatesConstraints(authorText, schema); - return { + result = { description: needsConstraints ? `${authorText} ${constraints}` : authorText, synthesized: false, }; + } else { + // A conventional name ("tripId", "coverImageUrl") reads as a real sentence; + // that beats the bare humanized name because it says what the name only + // hints at. Anything else gets the name plus whatever the schema proves. + const nameText = humanizeFieldName(name); + const pattern = patternSentence(name, schema, context?.noun); + if (pattern) { + const suffix = constraints || FORMAT_SENTENCES[schema.format ?? ""] || ""; + result = { description: suffix ? `${pattern} ${suffix}` : pattern, synthesized: true }; + } else { + const format = FORMAT_SENTENCES[schema.format ?? ""]; + // "Email. An email address." says one thing twice: when the format sentence + // already carries the noun, it is the draft on its own. + if ( + format && + constraints === format && + format.toLowerCase().includes(nameText.toLowerCase()) + ) { + result = { description: format, synthesized: true }; + } else { + const draft = + format && !constraints + ? `${nameText} (${format.replace(/^An?\s+/i, "").replace(/\.$/, "")}).` + : `${nameText}.${constraints ? ` ${constraints}` : ""}`; + result = { description: draft, synthesized: true }; + } + } } - // A conventional name ("tripId", "coverImageUrl") reads as a real sentence; - // that beats the bare humanized name because it says what the name only - // hints at. Anything else gets the name plus whatever the schema proves. - const nameText = humanizeFieldName(name); - const pattern = patternSentence(name, schema, context?.noun); - if (pattern) { - const suffix = constraints || FORMAT_SENTENCES[schema.format ?? ""] || ""; - return { description: suffix ? `${pattern} ${suffix}` : pattern, synthesized: true }; + // Only machine-drafted text is composed to fit the budget. Author text is + // preserved in full; verify warns on it instead of this layer silently + // dropping the author's words. + if (result.synthesized) { + result.description = fitBudget(result.description, FIELD_DESCRIPTION_MAX); } - const format = FORMAT_SENTENCES[schema.format ?? ""]; - // "Email. An email address." says one thing twice: when the format sentence - // already carries the noun, it is the draft on its own. - if (format && constraints === format && format.toLowerCase().includes(nameText.toLowerCase())) { - return { description: format, synthesized: true }; - } - const draft = - format && !constraints - ? `${nameText} (${format.replace(/^An?\s+/i, "").replace(/\.$/, "")}).` - : `${nameText}.${constraints ? ` ${constraints}` : ""}`; - return { description: draft, synthesized: true }; + return result; } /** @@ -380,7 +437,7 @@ const PHRASAL_VERBS = new Set(["sign-up", "sign-in", "sign-out", "log-in", "log- function returnShapeSentence(toolName: string, output: JsonSchema): string { const words = toolName.split("-"); - // The noun is what the verb leaves behind — and a phrasal verb is two words. + // The noun is what the verb leaves behind - and a phrasal verb is two words. const firstTwo = words.slice(0, 2).join("-"); const nounWords = PHRASAL_VERBS.has(firstTwo) ? words.slice(2) : words.slice(1); const nounPhrase = nounWords.join(" ").replace(/ by \w+$/, ""); @@ -415,7 +472,11 @@ export function describeCandidateTool(candidate: CandidateTool): void { const returns = candidate.outputSchema ? returnShapeSentence(candidate.name, candidate.outputSchema) : ""; - candidate.description = [sentence, returns].filter(Boolean).join(" "); + // No author text existed, so this is ours to compose: fit it. + candidate.description = fitBudget( + [sentence, returns].filter(Boolean).join(" "), + TOOL_DESCRIPTION_MAX, + ); candidate.descriptionSource = "generated-template"; return; } @@ -427,5 +488,7 @@ export function describeCandidateTool(candidate: CandidateTool): void { : ""; // The join is between sentences: the base earns its period first. const base = returns && !/[.!?]$/.test(normalized) ? `${normalized}.` : normalized; + // Author text plus a machine return sentence. The author's half is not + // shortened; verify warns if the total runs past the budget. candidate.description = [base, returns].filter(Boolean).join(" "); } diff --git a/packages/codegen/src/detect-app.ts b/packages/codegen/src/detect-app.ts index d113d03..5a8539a 100644 --- a/packages/codegen/src/detect-app.ts +++ b/packages/codegen/src/detect-app.ts @@ -2,12 +2,12 @@ * Web-app detection: where the generated tools should live. * * The tools are browser code, so they belong in whichever package *is* the - * web app — not next to the spec, and not wherever the command happened to + * web app - not next to the spec, and not wherever the command happened to * run. In a monorepo like: * * apps/ - * ├── server/ (has the openapi.json) - * └── web/ (has next in its package.json) ← tools go here + * |-- server/ (has the openapi.json) + * `-- web/ (has next in its package.json) <- tools go here * * detection means reading package.json files and looking for a browser * framework. One candidate: we use it and say so. Several: the CLI asks @@ -23,44 +23,6 @@ export interface WebApp { framework: "next" | "vite-react" | "nuxt" | "sveltekit" | "unknown"; } -/** - * Schema-module discovery for --suggest, mirroring findSpecs. Nobody should - * have to pass a file path to get proposals: we scan for modules that look - * like schema definitions, in the places they conventionally live. Returns - * module paths relative to cwd, likeliest first. - */ -const SCHEMA_MODULE_NAMES = /schemas?|models?|types|validation/i; -const SKIP_DIRS = new Set(["node_modules", "dist", ".git", ".next", "build", "out", "coverage"]); - -export async function findSchemaModules(cwd: string): Promise { - const found: string[] = []; - // Conventional homes first, so a monorepo's shared package wins over a - // deeply nested app file. - for (const base of ["packages", "src", "apps", "lib", "."]) { - await scanDir(join(cwd, base), base === "." ? "" : base, 0); - } - return [...new Set(found)]; - - async function scanDir(abs: string, rel: string, depth: number): Promise { - if (depth > 4) return; - let entries: import("node:fs").Dirent[]; - try { - entries = await readdir(abs, { withFileTypes: true }); - } catch { - return; - } - for (const entry of entries) { - if (entry.name.startsWith(".") || SKIP_DIRS.has(entry.name)) continue; - const entryRel = rel ? `${rel}/${entry.name}` : entry.name; - if (entry.isDirectory()) { - await scanDir(join(abs, entry.name), entryRel, depth + 1); - } else if (/\.(ts|mts|js|mjs)$/.test(entry.name) && SCHEMA_MODULE_NAMES.test(entry.name)) { - found.push(entryRel); - } - } - } -} - /** The validation libraries the schema source can read. */ const SCHEMA_LIBS = ["zod", "valibot", "arktype", "typebox"] as const; diff --git a/packages/codegen/src/dev/server.ts b/packages/codegen/src/dev/server.ts index 92319a8..07c9540 100644 --- a/packages/codegen/src/dev/server.ts +++ b/packages/codegen/src/dev/server.ts @@ -8,7 +8,7 @@ * shows is what a generate run would write. * * It exists only while the command is running, listens on localhost only, - * and nothing about it ever touches the user's app bundle — by design, so + * and nothing about it ever touches the user's app bundle - by design, so * this dev tool can never leak into production. */ @@ -200,7 +200,7 @@ function toUiTool( ? `schema: ${tool.source.ref}` : ""; // The dry run already holds every file's contents in memory, so the - // dashboard can show the real generated source per tool — the same + // dashboard can show the real generated source per tool - the same // progressive disclosure the site's demo has, against live output. const file = files.find((f) => basename(f.path) === `${tool.name}.webmcp.ts`); return { @@ -233,7 +233,7 @@ function toUiTool( * The direct "run it" test: call the endpoint the way the generated * execute() would, but server-side. Two honest limitations the UI states: * there is no browser session here (auth cookies do not apply), and the - * call needs an absolute base URL — the spec's servers entry or one the + * call needs an absolute base URL - the spec's servers entry or one the * developer types in. */ async function runEndpoint( diff --git a/packages/codegen/src/dev/ui.ts b/packages/codegen/src/dev/ui.ts index 1e878d5..2db2f4e 100644 --- a/packages/codegen/src/dev/ui.ts +++ b/packages/codegen/src/dev/ui.ts @@ -74,7 +74,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean } .app.detail-open .main { transform: translateX(0); } .main .placeholder { display: none; } - /* The back control is a bare chevron — a tap target, not a button. It + /* The back control is a bare chevron - a tap target, not a button. It sits inline at the left of the title row, so no vertical space is spent on navigation. */ .back-btn { @@ -140,7 +140,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean background: var(--surface); } /* The drag handle on the sidebar's right edge. Invisible until you - hover near it, then a 2px accent line — the Vercel/Linear idiom. */ + hover near it, then a 2px accent line - the Vercel/Linear idiom. */ .sidebar-resize { position: absolute; top: 0; @@ -402,7 +402,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean } /* The per-tool source disclosure: the generated file, revealed on demand. - A quiet row that expands into the code — the dashboard is the disclosure, + A quiet row that expands into the code - the dashboard is the disclosure, not a separate view. */ .source-disclosure { margin-bottom: 28px; @@ -753,7 +753,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean ::-webkit-scrollbar-thumb:hover { background: var(--ghost); } /* Narrow-screen content density. This block comes after the base .detail - so it actually wins on source order — an earlier media query lost to the + so it actually wins on source order - an earlier media query lost to the desktop rule, which is why the padding never changed. */ @media (max-width: 640px) { .detail { padding: 14px 16px; max-width: none; } @@ -766,7 +766,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean ? ` /* Scoped mode: mounted inside a shadow root on the marketing site, where document-level selectors never match and vh/dvh would measure the page - viewport, not the host — which is exactly what clipped the demo's scroll + viewport, not the host - which is exactly what clipped the demo's scroll region before. This block comes LAST so it overrides the base rules: :host plays the body role and the app fills it, not the viewport. */ :host { @@ -784,7 +784,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean position: relative; overflow: hidden; } - /* The demo's default sidebar width — narrower than the real dashboard's + /* The demo's default sidebar width - narrower than the real dashboard's 320px, so the detail pane gets the room in the embedded frame. The drag handle sets an inline width, which still wins over this; the mobile full-width sidebar rule carries !important and is unaffected. */ @@ -815,7 +815,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean
- +
@@ -826,7 +826,7 @@ export function dashboardHtml(embeddedState?: UiState, opts?: { scoped?: boolean

Select a tool to view details

- ↑ ↓ to navigate  ·  ⌘K to search + Up Down to navigate  -  CmdK to search

@@ -960,7 +960,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; ].filter(Boolean).join(""); var findings = (tool.findings || []).map(function (finding) { - var icon = finding.level === "error" ? "✖" : "⚠"; + var icon = finding.level === "error" ? "x" : "!"; return '
' + icon + "" + esc(finding.message) + "
"; }).join(""); @@ -1050,7 +1050,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; '' + 'Saved' + "" + - '
Agents pick tools by this text. Saved to .webmcp-codegen.json, so it survives regeneration. ⌘S to save.
' + + '
Agents pick tools by this text. Saved to .webmcp-codegen.json, so it survives regeneration. CmdS to save.
' + "" + (fieldRows @@ -1079,7 +1079,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; '

Run this tool

server-side, no browser session
' + '
' + (tool.requiresAuth - ? '
⚠ This endpoint requires a browser session. The dashboard runs server-side, so you will get a 401. Test it in Chrome DevTools where you are signed in.
' + ? '
! This endpoint requires a browser session. The dashboard runs server-side, so you will get a 401. Test it in Chrome DevTools where you are signed in.
' : "") + '' + (fields || '
This tool takes no inputs.
') + @@ -1235,7 +1235,7 @@ var EMBEDDED_STATE = ${embeddedState ? JSON.stringify(embeddedState) : "null"}; } if ((event.metaKey || event.ctrlKey) && event.key === "s") { event.preventDefault(); - // ⌘S saves whatever is being edited: an open field row, else the + // CmdS saves whatever is being edited: an open field row, else the // tool description. if (document.getElementById("field-edit-input")) saveFieldEdit(); else saveDescription(); diff --git a/packages/codegen/src/group.test.ts b/packages/codegen/src/group.test.ts new file mode 100644 index 0000000..3cdca5d --- /dev/null +++ b/packages/codegen/src/group.test.ts @@ -0,0 +1,135 @@ +import { describe, expect, it } from "vitest"; +import { groupHandshakes } from "./group.js"; +import { generatedRegion } from "./outputs/tools-templates.js"; +import type { CandidateTool, ReviewedTool } from "./types.js"; + +function post( + overrides: Partial & { name: string; pathTemplate: string }, +): CandidateTool { + return { + id: `POST ${overrides.pathTemplate}`, + source: { kind: "openapi", ref: `POST ${overrides.pathTemplate}` }, + inputSchema: { type: "object", properties: {} }, + inputTypeName: "Input", + httpMethod: "POST", + paramLocations: { path: [], query: [], body: [] }, + sideEffect: "write", + requiresAuth: false, + description: "Does a thing.", + descriptionSource: "openapi-summary", + ...overrides, + }; +} + +/** The beenthere upload handshake, faithfully. */ +function uploadPair(): CandidateTool[] { + return [ + post({ + name: "create-media-request-upload", + pathTemplate: "/v1/media/request-upload", + paramLocations: { path: [], query: [], body: ["fileName"] }, + inputSchema: { + type: "object", + properties: { fileName: { type: "string", description: "The file's name." } }, + required: ["fileName"], + }, + outputSchema: { + type: "object", + properties: { uploadId: { type: "string" }, url: { type: "string" } }, + }, + }), + post({ + name: "complete-media-upload", + pathTemplate: "/v1/media/uploads/{uploadId}/complete", + paramLocations: { path: ["uploadId"], query: [], body: [] }, + inputSchema: { + type: "object", + properties: { uploadId: { type: "string", description: "The upload to complete." } }, + required: ["uploadId"], + }, + outputSchema: { type: "object", properties: { media: { type: "object" } } }, + }), + ]; +} + +describe("groupHandshakes", () => { + it("merges a request/complete pair into one withheld, thread-wired proposal", () => { + const { tools, notes } = groupHandshakes(uploadPair()); + + // Members stay; the merged tool is appended. + expect(tools).toHaveLength(3); + const merged = tools.find((tool) => tool.name === "upload-media"); + expect(merged).toBeDefined(); + expect(merged?.compose?.threaded).toEqual({ uploadId: "uploadId" }); + + // The merged input drops the threaded field, keeps the real input. + expect(Object.keys(merged?.inputSchema.properties ?? {})).toEqual(["fileName"]); + expect(merged?.inputSchema.required).toEqual(["fileName"]); + + // The result is the SECOND call's response. + expect(merged?.outputSchema?.properties).toHaveProperty("media"); + + // The report names the proposal and how to adopt it. + expect(notes[0]).toContain( + "Grouped create-media-request-upload + complete-media-upload into upload-media", + ); + expect(notes[0]).toContain("starts withheld"); + }); + + it("skips the pair when the second call's path param can't be threaded", () => { + const pair = uploadPair(); + pair[0]!.outputSchema = { type: "object", properties: { token: { type: "string" } } }; + const { tools, notes } = groupHandshakes(pair); + expect(tools).toHaveLength(2); + expect(notes[0]).toContain("path params the first response doesn't provide"); + }); + + it("does not pair across different resources", () => { + const pair = uploadPair(); + pair[1]!.pathTemplate = "/v1/videos/uploads/{uploadId}/complete"; + pair[1]!.name = "complete-videos-upload"; + const { tools } = groupHandshakes(pair); + expect(tools).toHaveLength(2); + }); + + it("does not pair GETs or lone POSTs", () => { + const readsOnly = [post({ name: "request-data-export", pathTemplate: "/v1/data/request" })]; + expect(groupHandshakes(readsOnly).tools).toHaveLength(1); + + const getPair = uploadPair().map((tool) => ({ ...tool, httpMethod: "GET" as const })); + expect(groupHandshakes(getPair).tools).toHaveLength(2); + }); + + it("falls back to a free name when the natural one is taken", () => { + const pair = uploadPair(); + const blocker = post({ name: "upload-media", pathTemplate: "/v1/media/other" }); + const { tools } = groupHandshakes([...pair, blocker]); + const merged = tools.find((tool) => tool.compose); + expect(merged?.name).not.toBe("upload-media"); + expect(merged?.name).toBeTruthy(); + }); +}); + +describe("composed tool templates", () => { + it("emits a two-call fetchX that threads the first response into the second call", () => { + const { tools } = groupHandshakes(uploadPair()); + const merged = tools.find((tool) => tool.compose); + const region = generatedRegion({ + ...merged, + enabledByDefault: false, + withheld: true, + riskTier: "write-confirm", + hints: { readOnlyHint: false, untrustedContentHint: false }, + piiInOutput: [], + endpointRole: "endpoint", + } as unknown as ReviewedTool); + + expect(region).toContain("export async function fetchUploadMedia("); + expect(region).toContain('const firstResult = (await callApi("/v1/media/request-upload"'); + // biome-ignore lint/suspicious/noTemplateCurlyInString: asserting the emitted template literal, not interpolating in this test. + expect(region).toContain("${firstResult.uploadId}/complete"); + expect(region).toContain("body: { fileName: input.fileName }"); + // The merged tool is a write: the confirmation gate applies. + expect(region).toContain("requestUserConfirmation("); + }); +}); diff --git a/packages/codegen/src/group.ts b/packages/codegen/src/group.ts new file mode 100644 index 0000000..3179111 --- /dev/null +++ b/packages/codegen/src/group.ts @@ -0,0 +1,216 @@ +/** + * The grouping step: find the endpoints that are one action split by API + * shape, and emit the single coarse tool an agent should see instead. + * + * The canonical case is the upload handshake: POST /media/request-upload + + * POST /media/uploads/{uploadId}/complete is not two decisions, it's one + * upload split in two calls. An agent that must orchestrate the pair gets + * hallucination room ("what's an uploadId again?"); one upload-media tool + * closes it. + * + * Restraint is the whole game here (see the direction doc: naive 1:1 mapping + * is the failure, but so is inventing cleverness). A pair merges ONLY when + * every line holds deterministically: + * + * - both are POST endpoints under the same first path segment (one resource) + * - the left name carries a begin-verb (request/begin/initiate/prepare/start) + * and the right name an end-verb (complete/commit/confirm/finish/end) + * - the right name carries the left's final token (the shared noun: + * "upload" in request-upload + complete-media-upload) + * - the shared noun token is at least 3 letters ("do" pairs nothing) + * + * Threading (how the merged call feeds the second request from the first + * response) is exact-name only: the right side's path params must each match + * a property on the left's response schema (`{uploadId}` <- `uploadId`). If a + * path param can't be threaded, the pair is skipped - guessing data flow is + * how plausible garbage gets shipped. + * + * The merged tool is withheld like any write: the merge is a proposal the + * developer adopts by enabling it. Members are left untouched (they're + * withheld writes already), so adopting never means losing the alternative. + */ + +import pluralize from "pluralize"; +import { pascalCase } from "./json-schema.js"; +import type { CandidateTool, JsonSchema } from "./types.js"; + +/** Verbs that start a handshake, and the verbs allowed to end it. */ +const BEGIN_VERBS = new Set(["request", "begin", "initiate", "prepare", "start"]); +const END_VERBS = new Set(["complete", "commit", "confirm", "finish", "end"]); + +/** One HTTP call of a composed tool. */ +export interface ComposedCall { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + /** The input type of the endpoint this call came from. */ +} + +/** What a merged tool needs to emit its two-call request. */ +export interface ComposedPlan { + first: ComposedCall; + second: ComposedCall; + /** Second-call fields filled from the first response, keyed by field name. */ + threaded: Record; +} + +export interface GroupResult { + tools: CandidateTool[]; + notes: string[]; +} + +/** The response property names a call's result exposes for threading. */ +function responseFields(tool: CandidateTool): Set { + return new Set(Object.keys(tool.outputSchema?.properties ?? {})); +} + +/** True when the right endpoint's path params can all be filled from left response fields. */ +function threadingFor( + left: CandidateTool, + right: CandidateTool, +): Record | undefined { + const available = responseFields(left); + const threaded: Record = {}; + for (const param of right.paramLocations?.path ?? []) { + if (!available.has(param)) return undefined; + threaded[param] = param; + } + return threaded; +} + +/** The verb tokens of a name, in order. */ +function tokensOf(name: string): string[] { + return name.split("-").filter(Boolean); +} + +function firstPathSegment(pathTemplate: string | undefined): string | undefined { + // Version prefixes ("v1", "v2") carry no resource identity of their own. + return pathTemplate + ?.split("/") + .filter(Boolean) + .find((segment) => !/^v\d+$/.test(segment)); +} + +/** Try the candidate's preferred names in order; return the first free one. */ +function pickMergedName(noun: string, resource: string, taken: Set): string | undefined { + const resourceSingular = pluralize.singular(resource); + const candidates = + noun === resourceSingular || noun === resource + ? [noun, `${noun}-flow`] + : [`${noun}-${resourceSingular}`, `${noun}-${resource}`, `${noun}-flow`]; + return candidates.find((candidate) => !taken.has(candidate)); +} + +/** Merge two schemas' properties; left wins on name conflict. */ +function mergedInputSchema( + left: CandidateTool, + right: CandidateTool, + threaded: Record, +): JsonSchema { + const properties: Record = { + ...(right.inputSchema.properties ?? {}), + ...(left.inputSchema.properties ?? {}), + }; + for (const field of Object.keys(threaded)) delete properties[field]; + const required = new Set([ + ...(left.inputSchema.required ?? []), + ...(right.inputSchema.required ?? []), + ]); + for (const field of Object.keys(threaded)) required.delete(field); + return { + type: "object", + properties, + ...(required.size > 0 ? { required: [...required].sort() } : {}), + }; +} + +/** + * Detect handshake pairs among named candidates and append merged proposals. + * Members stay in the list exactly as they were; the merged tool arrives + * marked so the safety layer gives it the write treatment its members have. + */ +export function groupHandshakes(candidates: CandidateTool[]): GroupResult { + const notes: string[] = []; + const added: CandidateTool[] = []; + const taken = new Set(candidates.map((tool) => tool.name)); + + const posts = candidates.filter( + (tool) => tool.httpMethod === "POST" && tool.pathTemplate && tool.paramLocations, + ); + + const consumed = new Set(); + for (const left of posts) { + if (consumed.has(left.id)) continue; + const leftTokens = tokensOf(left.name); + if (!leftTokens.some((token) => BEGIN_VERBS.has(token))) continue; + const noun = leftTokens[leftTokens.length - 1]; + if (!noun || noun.length < 3) continue; + const resource = firstPathSegment(left.pathTemplate); + if (!resource) continue; + + const right = posts.find( + (candidate) => + candidate !== left && + !consumed.has(candidate.id) && + firstPathSegment(candidate.pathTemplate) === resource && + tokensOf(candidate.name).some((token) => END_VERBS.has(token)) && + tokensOf(candidate.name).includes(noun), + ); + if (!right) continue; + + const threaded = threadingFor(left, right); + if (threaded === undefined) { + notes.push( + `${left.name} + ${right.name} look like one flow, but "${right.name}" takes path params the first response doesn't provide - left as separate tools.`, + ); + continue; + } + + const name = pickMergedName(noun, resource, taken); + if (!name) { + notes.push( + `${left.name} + ${right.name} look like one flow, but every merged name collided - left as separate tools.`, + ); + continue; + } + + taken.add(name); + consumed.add(left.id); + consumed.add(right.id); + + const resourceSingular = pluralize.singular(resource); + added.push({ + id: `${left.id} + ${right.id}`, + name, + source: { kind: "openapi", ref: `${left.source.ref} + ${right.source.ref}` }, + inputSchema: mergedInputSchema(left, right, threaded), + outputSchema: right.outputSchema, + inputTypeName: `${pascalCase(name)}Input`, + sideEffect: "write", + requiresAuth: left.requiresAuth || right.requiresAuth, + description: `Complete the ${resourceSingular} ${noun}. One action the API split into two calls; they run in order. Returns the completed ${noun}.`, + descriptionSource: "generated-template", + compose: { + first: { + httpMethod: left.httpMethod as NonNullable, + pathTemplate: left.pathTemplate as string, + paramLocations: left.paramLocations as ComposedCall["paramLocations"], + serverUrl: left.serverUrl, + }, + second: { + httpMethod: right.httpMethod as NonNullable, + pathTemplate: right.pathTemplate as string, + paramLocations: right.paramLocations as ComposedCall["paramLocations"], + serverUrl: right.serverUrl, + }, + threaded, + }, + }); + notes.push( + `Grouped ${left.name} + ${right.name} into ${name} - one action the API split in two calls. It starts withheld like its members; enable ${name} instead of the pair when you're satisfied.`, + ); + } + + return { tools: [...candidates, ...added], notes }; +} diff --git a/packages/codegen/src/index.ts b/packages/codegen/src/index.ts index 45a6adc..206e8b6 100644 --- a/packages/codegen/src/index.ts +++ b/packages/codegen/src/index.ts @@ -2,7 +2,7 @@ * webmcp-codegen public API. * * Most people only ever need `defineConfig`. Sources and outputs live - * behind their own subpaths ("@webmcp-stack/codegen/sources", "…/outputs") + * behind their own subpaths ("@webmcp-stack/codegen/sources", ".../outputs") * so the top-level import stays small. */ diff --git a/packages/codegen/src/json-schema.ts b/packages/codegen/src/json-schema.ts index 876ded0..6128bc7 100644 --- a/packages/codegen/src/json-schema.ts +++ b/packages/codegen/src/json-schema.ts @@ -56,7 +56,7 @@ export function deref(schema: JsonSchema, spec: unknown): JsonSchema { * self-contained JSON Schema. The browser has no idea what * "#/components/schemas/Order" means, so refs must not survive codegen. * - * Recursive models (Order → LineItem → Order) would loop forever, so a ref + * Recursive models (Order -> LineItem -> Order) would loop forever, so a ref * that points back to one of its own ancestors resolves to a plain object * with a note instead. The tool schema stays finite and honest. */ @@ -152,7 +152,7 @@ export function jsonSchemaToTs(schema: JsonSchema, spec: unknown): string { } } -/** "get-order-status" → "GetOrderStatus" (for generated type names). */ +/** "get-order-status" -> "GetOrderStatus" (for generated type names). */ export function pascalCase(name: string): string { return name .split(/[-_]/) diff --git a/packages/codegen/src/llm.test.ts b/packages/codegen/src/llm.test.ts deleted file mode 100644 index 4b271ac..0000000 --- a/packages/codegen/src/llm.test.ts +++ /dev/null @@ -1,252 +0,0 @@ -import { describe, expect, it } from "vitest"; -import { hostedLlmProvider, resolveLlmProvider, runLlmLayer } from "./llm.js"; -import type { LlmProvider, ReviewedTool } from "./types.js"; - -/** A provider that records what it was asked and answers per task. */ -function mockProvider(answers: Record = {}) { - const calls: { task: string; prompt: string }[] = []; - const provider: LlmProvider = { - name: "mock", - async complete(task, prompt) { - calls.push({ task, prompt }); - return answers[task] ?? "{}"; - }, - }; - return { provider, calls }; -} - -function tool(overrides: Partial = {}): ReviewedTool { - return { - id: "GET /x", - name: "get-x", - source: { kind: "openapi", ref: "GET /x" }, - inputSchema: { type: "object", properties: {}, required: [] }, - inputTypeName: "GetXInput", - httpMethod: "GET", - sideEffect: "read", - endpointRole: "endpoint", - enabledByDefault: true, - withheld: false, - requiresAuth: false, - riskTier: "safe-read", - hints: { - readOnlyHint: true, - destructiveHint: false, - idempotentHint: true, - untrustedContentHint: false, - }, - piiInOutput: [], - description: "A fine description", - descriptionSource: "openapi-summary", - ...overrides, - }; -} - -describe("resolveLlmProvider", () => { - it("is off without the config key, even with an env key set", () => { - // An OPENAI_API_KEY in the environment must not turn a generate run into - // a network call nobody asked for; the config key is the opt-in. - expect(resolveLlmProvider(undefined, { OPENAI_API_KEY: "sk-test" })).toBeUndefined(); - }); - - it("uses the custom provider when one is given", () => { - const { provider } = mockProvider(); - expect(resolveLlmProvider({ provider })?.name).toBe("mock"); - }); - - it("builds the built-in provider from a config key, then env keys", () => { - expect(resolveLlmProvider({ apiKey: "sk-config" }, {})?.name).toContain("openai-compatible"); - expect(resolveLlmProvider({}, { WEBMCP_LLM_API_KEY: "sk-env" })?.name).toContain( - "openai-compatible", - ); - expect(resolveLlmProvider({}, { OPENAI_API_KEY: "sk-env" })?.name).toContain( - "openai-compatible", - ); - expect(resolveLlmProvider({}, {})).toBeUndefined(); - }); - - it("the hosted tier points at our proxy, key never in the package", async () => { - const provider = hostedLlmProvider(); - expect(provider.name).toContain("openai-compatible"); - // The wire call goes to our site; the bearer is a placeholder the proxy ignores. - let seenUrl = ""; - let seenAuth = ""; - const fetchSpy = async (input: RequestInfo | URL, init?: RequestInit) => { - seenUrl = String(input); - seenAuth = String((init?.headers as Record)?.authorization ?? ""); - return new Response(JSON.stringify({ choices: [{ message: { content: "{}" } }] }), { - status: 200, - }); - }; - const realFetch = globalThis.fetch; - globalThis.fetch = fetchSpy as typeof fetch; - try { - await provider.complete("describe", "prompt", "system"); - } finally { - globalThis.fetch = realFetch; - } - expect(seenUrl).toBe("https://webmcp-stack.vercel.app/api/llm/chat/completions"); - expect(seenAuth).toBe("Bearer hosted"); - }); -}); - -describe("runLlmLayer", () => { - it("proposes drafts only for machine-written text, never author text", async () => { - const { provider, calls } = mockProvider({ - describe: JSON.stringify({ - description: "Create a trip and open it in the editor.", - fields: { minutes: "Minutes worked on this task. A number from 30 to 600." }, - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [ - tool({ name: "well-described" }), // author text: never sent - tool({ - name: "create-trip", - description: "create-trip", - descriptionSource: "generated-template", - synthesizedFields: ["minutes"], - }), - ], - findings: [], - }, - ); - - // Only the needy tool was asked about. - expect(calls.filter((call) => call.task === "describe")).toHaveLength(1); - expect(calls[0]?.prompt).toContain("create-trip"); - expect(suggestions.some((s) => s.message.includes("Create a trip and open it"))).toBe(true); - expect(suggestions.some((s) => s.field === "minutes" && s.message.includes("30 to 600"))).toBe( - true, - ); - }); - - it("suggests a producer for fields the audit flagged as unproducible", async () => { - const { provider } = mockProvider({ - relationship: JSON.stringify({ producer: "search-places" }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [tool({ name: "create-trip" }), tool({ name: "search-places" })], - findings: [ - { - level: "warning", - tool: "create-trip", - message: - 'Field "locationObject" says it comes from the "search-places" tool, but no such tool exists in this run.', - }, - ], - }, - ); - expect(suggestions.some((s) => s.message.includes('"search-places"'))).toBe(true); - }); - - it("never names a producer that does not exist in the run", async () => { - const { provider } = mockProvider({ - relationship: JSON.stringify({ producer: "hallucinated-tool" }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [tool({ name: "create-trip" })], - findings: [ - { - level: "warning", - tool: "create-trip", - message: 'Field "locationObject" ... but no such tool exists in this run.', - }, - ], - }, - ); - expect(suggestions.filter((s) => s.task === "relationship")).toHaveLength(0); - }); - - it("surfaces semantic contradictions as suggestions, not findings", async () => { - const { provider } = mockProvider({ - "semantic-review": JSON.stringify({ - contradictions: [{ tool: "get-x", issue: 'Says "fetch" but the schema writes.' }], - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { tools: [tool({ name: "get-x" })], findings: [] }, - ); - expect( - suggestions.some((s) => s.task === "semantic-review" && s.message.includes("fetch")), - ).toBe(true); - }); - - it("proposes tools for --suggest exports", async () => { - const { provider } = mockProvider({ - suggest: JSON.stringify({ - tools: [{ name: "create-trip", description: "Create a trip.", write: true }], - }), - }); - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - tools: [], - findings: [], - suggestExports: [{ name: "CreateTripInput", schemaText: '{"type":"object"}' }], - }, - ); - expect(suggestions.some((s) => s.task === "suggest" && s.message.includes("create-trip"))).toBe( - true, - ); - }); - - it("asks each distinct question once per run, however many tools share it", async () => { - const { provider, calls } = mockProvider({ - describe: JSON.stringify({ description: "Draft." }), - }); - const twin = { - description: "same", - descriptionSource: "generated-template" as const, - synthesizedFields: [] as string[], - inputSchema: { type: "object", properties: {}, required: [] }, - }; - await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { - // Two tools with byte-identical questions hash to one provider call. - tools: [tool({ name: "same", ...twin }), tool({ name: "same", ...twin })], - findings: [], - }, - ); - expect(calls.filter((call) => call.task === "describe")).toHaveLength(1); - }); - - it("turns a provider failure into a note, never a run failure", async () => { - const broken: LlmProvider = { - name: "broken", - async complete() { - throw new Error("HTTP 500"); - }, - }; - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider: broken } }, - { - tools: [tool({ name: "t", description: "t", descriptionSource: "generated-template" })], - findings: [], - }, - ); - expect( - suggestions.some((s) => s.message.includes("failed") && s.message.includes("unaffected")), - ).toBe(true); - }); - - it("parses completions that wrap JSON in prose or fences", async () => { - const { provider } = mockProvider({ - "semantic-review": 'Here you go:\n```json\n{"contradictions": []}\n```\nHope that helps!', - }); - // No throw, no suggestions, no drama. - const suggestions = await runLlmLayer( - { sources: [], outputs: [], llm: { provider } }, - { tools: [tool()], findings: [] }, - ); - expect(suggestions.filter((s) => s.task === "semantic-review")).toHaveLength(0); - }); -}); diff --git a/packages/codegen/src/llm.ts b/packages/codegen/src/llm.ts deleted file mode 100644 index 4f4df7a..0000000 --- a/packages/codegen/src/llm.ts +++ /dev/null @@ -1,398 +0,0 @@ -/** - * The LLM layer: advisory, always optional, never load-bearing. - * - * Why this file is shaped the way it is: the codegen's trust model rests on - * the audit being reproducible. Same input, same findings, same exit codes, - * on a laptop or in CI. A model breaks that, so the boundary here is a hard - * rule, not a guideline: - * - * THE MODEL PROPOSES. THE DEVELOPER DISPOSES. - * - * Concretely, in this version: - * - proposals surface only as `◦` report lines. Nothing is written to a - * file, because the acceptance surface (dashboard accept/reject) is a - * deliberate follow-up; without a place to dispose, nothing is applied. - * - the layer never classifies side effects or risk tiers, never changes - * exit codes, and never blocks a run. A missing key means off; a failing - * provider means one warning line and the deterministic run continues. - * - it runs only on what is still machine-written after the deterministic - * layers (template tool descriptions, synthesized field text). Author - * text is never sent back for "improvement". - * - * The prompts below are the shipped defaults, benchmarked against the WebMCP - * docs' examples. `llm.prompts` in the config overrides them per task: the - * developer owns the words going into their agent's prompt. - */ - -import { createHash } from "node:crypto"; -import type { - AuditFinding, - CodegenConfig, - LlmOptions, - LlmProvider, - LlmSuggestion, - LlmTask, - ReviewedTool, -} from "./types.js"; - -/** What the layer needs from the rest of the pipeline. */ -export interface LlmContext { - tools: ReviewedTool[]; - findings: AuditFinding[]; - /** Module exports for the `suggest` task: name + JSON Schema text. */ - suggestExports?: { name: string; schemaText: string }[]; -} - -const DEFAULT_PROMPTS: Record = { - describe: [ - "You write WebMCP tool descriptions. Agents pick tools by these words and fill", - "inputs from them, so be concrete, imperative, and one sentence per item. State", - "what the tool does and what each field means; mention constraints you are given.", - 'Answer with JSON only: { "description"?: string, "fields": { "": string } }.', - ].join(" "), - relationship: [ - "You are given a tool field whose description says its value comes from another", - "tool, plus the list of tools that exist. Name the likeliest producer tool, or", - 'null if none fits. Answer with JSON only: { "producer": string | null }.', - ].join(" "), - "semantic-review": [ - "You review WebMCP tool descriptions against their input schemas. Flag only real", - 'contradictions (a "fetch" description on a writing schema, a field described as', - "the opposite of its type). Silence is a good outcome. Answer with JSON only:", - '{ "contradictions": [{ "tool": string, "issue": string }] }.', - ].join(" "), - suggest: [ - "You are given exported validation schemas from a web app. Propose which should", - "become agent-facing WebMCP tools: user-meaningful actions, not plumbing. For each,", - "name (kebab-case, verb-first), one-sentence description, and whether it writes.", - 'Answer with JSON only: { "tools": [{ "name": string, "description": string, "write": boolean }] }.', - ].join(" "), -}; - -/** - * Resolve the configured provider, or undefined (layer off). Explicit opt-in - * is the config key: an OPENAI_API_KEY in the environment must not turn a - * `generate` run into a network call the developer never asked for. - * (`generate --suggest` is the explicit command, and honors env keys.) - */ -export function resolveLlmProvider( - options: LlmOptions | undefined, - env: NodeJS.ProcessEnv = process.env, -): LlmProvider | undefined { - if (!options) return undefined; - if (options.provider) return options.provider; - const apiKey = options.apiKey ?? env.WEBMCP_LLM_API_KEY ?? env.OPENAI_API_KEY; - if (!apiKey) return undefined; - return openAiCompatibleProvider( - apiKey, - options.baseUrl ?? "https://api.openai.com/v1", - options.model ?? "gpt-4o-mini", - ); -} - -/** - * The free hosted tier: our own proxy holds the key server-side, so a - * developer with no provider account can still try the LLM layer. Rate - * limits live on the proxy; the CLI just points at it. The proxy ignores - * the bearer — the real key never leaves the server. - */ -export const HOSTED_LLM_URL = "https://webmcp-stack.vercel.app/api/llm"; - -export function hostedLlmProvider(): LlmProvider { - return openAiCompatibleProvider("hosted", HOSTED_LLM_URL, "openai/gpt-4o-mini"); -} - -/** - * The built-in provider: any OpenAI-compatible chat-completions endpoint, - * via plain fetch. Zero dependencies is a feature of this package, so the - * wire format is spelled out rather than imported. - */ -function openAiCompatibleProvider(apiKey: string, baseUrl: string, model: string): LlmProvider { - return { - name: `openai-compatible (${model})`, - async complete(task, prompt, system) { - const response = await fetch(`${baseUrl}/chat/completions`, { - method: "POST", - headers: { - "content-type": "application/json", - authorization: `Bearer ${apiKey}`, - }, - body: JSON.stringify({ - model, - messages: [ - // The developer's per-task override wins over the shipped default; - // config.llm.prompts exists so teams can tune these. - { role: "system", content: system ?? DEFAULT_PROMPTS[task] }, - { role: "user", content: prompt }, - ], - temperature: 0, - }), - // A hung endpoint must never hang a generate run. - signal: AbortSignal.timeout(30_000), - }); - if (!response.ok) { - throw new Error(`LLM endpoint answered HTTP ${response.status}`); - } - const body = (await response.json()) as { - choices?: { message?: { content?: string } }[]; - }; - const content = body.choices?.[0]?.message?.content; - if (!content) throw new Error("LLM endpoint returned no content"); - return content; - }, - }; -} - -/** - * In-run cache, keyed by content hash. Many fields share a shape (every - * "name" string in a spec), so identical questions are asked once per run. - * Deliberately not a disk cache: a cache file in the user's repo is noise, - * and stale-across-runs is a bug class this sidesteps entirely. - */ -function cached(provider: LlmProvider): LlmProvider { - const seen = new Map>(); - return { - name: provider.name, - complete(task, prompt, system) { - // The system prompt is part of the question's identity: the same - // question under two different overrides must not share an answer. - const key = createHash("sha256") - .update(task) - .update("\0") - .update(system ?? "") - .update("\0") - .update(prompt) - .digest("hex"); - let pending = seen.get(key); - if (!pending) { - pending = provider.complete(task, prompt, system); - seen.set(key, pending); - } - return pending; - }, - }; -} - -/** Extract the JSON object from a completion that may wrap it in prose. */ -function parseCompletionJson(text: string): Record | undefined { - const direct = tryParse(text); - if (direct) return direct; - const fenced = /```(?:json)?\s*(\{[\s\S]*?\})\s*```/.exec(text); - if (fenced) return tryParse(fenced[1] as string); - const braces = /\{[\s\S]*\}/.exec(text); - return braces ? tryParse(braces[0]) : undefined; -} - -function tryParse(text: string): Record | undefined { - try { - const value: unknown = JSON.parse(text); - return value && typeof value === "object" ? (value as Record) : undefined; - } catch { - return undefined; - } -} - -/** - * Run every enabled touchpoint and collect proposals. Never throws: a broken - * provider yields one suggestion line explaining the failure, and the rest of - * the run is exactly the deterministic one. - */ -export async function runLlmLayer( - config: CodegenConfig, - context: LlmContext, - env?: NodeJS.ProcessEnv, - /** When the caller already resolved a provider (the interactive key - * chooser), pass it so the layer does not re-resolve from config/env. */ - explicitProvider?: LlmProvider, -): Promise { - const configured = explicitProvider ?? resolveLlmProvider(config.llm, env); - if (!configured) return []; - const provider = cached(configured); - const prompts = { ...DEFAULT_PROMPTS, ...config.llm?.prompts }; - const suggestions: LlmSuggestion[] = []; - - const jobs: Promise[] = [ - proposeDescriptions(provider, prompts.describe, context.tools, suggestions), - proposeProducers(provider, prompts.relationship, context, suggestions), - reviewSemantics(provider, prompts["semantic-review"], context.tools, suggestions), - ]; - if (context.suggestExports && context.suggestExports.length > 0) { - jobs.push(proposeTools(provider, prompts.suggest, context.suggestExports, suggestions)); - } - - // Each job catches its own failures; this allSettled is the belt to their - // suspenders, because an LLM failure must never surface as a run failure. - await Promise.allSettled(jobs); - return suggestions; -} - -/** - * Draft prose for what the deterministic layers left machine-written: - * template tool descriptions and synthesized field text. Author text is - * never in scope. - */ -async function proposeDescriptions( - provider: LlmProvider, - prompt: string, - tools: ReviewedTool[], - suggestions: LlmSuggestion[], -): Promise { - const needy = tools.filter( - (tool) => - tool.descriptionSource === "generated-template" || (tool.synthesizedFields ?? []).length > 0, - ); - for (const tool of needy) { - const fields = (tool.synthesizedFields ?? []) - .map((name) => { - const schema = tool.inputSchema.properties?.[name]; - return `"${name}": ${JSON.stringify(schema)}`; - }) - .join(", "); - const question = - `Tool "${tool.name}" (currently described as "${tool.description}"). ` + - (fields ? `Fields needing text: { ${fields} }.` : "No fields need text."); - try { - const answer = parseCompletionJson(await provider.complete("describe", question, prompt)); - if (typeof answer?.description === "string" && answer.description.trim()) { - suggestions.push({ - task: "describe", - tool: tool.name, - message: `${tool.name}: draft description: "${answer.description.trim()}"`, - }); - } - const fieldDrafts = answer?.fields; - if (fieldDrafts && typeof fieldDrafts === "object") { - for (const [field, text] of Object.entries(fieldDrafts)) { - if (typeof text === "string" && text.trim()) { - suggestions.push({ - task: "describe", - tool: tool.name, - field, - message: `${tool.name}.${field}: draft: "${text.trim()}"`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("describe", tool.name, error)); - return; // One failure is enough; hammering a broken endpoint helps nobody. - } - } -} - -/** - * Where the deterministic rule found a field referencing a producer tool that - * does not exist, ask the model whether an existing tool could produce it. - * A suggestion here is inference, so it is always a proposal, never a finding. - */ -async function proposeProducers( - provider: LlmProvider, - prompt: string, - context: LlmContext, - suggestions: LlmSuggestion[], -): Promise { - const gaps = context.findings.filter( - (finding) => finding.tool && finding.message.includes("no such tool"), - ); - const toolNames = context.tools.map((tool) => tool.name); - for (const gap of gaps) { - const question = - `Field in tool "${gap.tool}": ${gap.message}\n` + - `Existing tools: ${toolNames.join(", ") || "(none)"}.`; - try { - const answer = parseCompletionJson(await provider.complete("relationship", question, prompt)); - const producer = answer?.producer; - if (typeof producer === "string" && toolNames.includes(producer)) { - suggestions.push({ - task: "relationship", - tool: gap.tool, - message: `${gap.tool}: "${producer}" looks like it could produce that value. Link them?`, - }); - } - } catch (error) { - suggestions.push(failureNote("relationship", gap.tool, error)); - return; - } - } -} - -/** - * A second pair of eyes on author-written descriptions: contradictions with - * the schema only. Advisory even when enabled; nothing here blocks anything. - */ -async function reviewSemantics( - provider: LlmProvider, - prompt: string, - tools: ReviewedTool[], - suggestions: LlmSuggestion[], -): Promise { - const described = tools - .filter((tool) => tool.descriptionSource !== "generated-template") - .slice(0, 25); // Cost guard: one bounded call, not a novel per spec. - if (described.length === 0) return; - const question = described - .map((tool) => { - const fields = Object.keys(tool.inputSchema.properties ?? {}).join(", "); - return `"${tool.name}": "${tool.description}" (fields: ${fields})`; - }) - .join("\n"); - try { - const answer = parseCompletionJson( - await provider.complete("semantic-review", question, prompt), - ); - const contradictions = answer?.contradictions; - if (Array.isArray(contradictions)) { - for (const item of contradictions) { - const entry = item as { tool?: unknown; issue?: unknown }; - if (typeof entry.tool === "string" && typeof entry.issue === "string") { - suggestions.push({ - task: "semantic-review", - tool: entry.tool, - message: `${entry.tool}: ${entry.issue}`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("semantic-review", undefined, error)); - } -} - -/** `generate --suggest `: which exported schemas are worth a tool? */ -async function proposeTools( - provider: LlmProvider, - prompt: string, - exports_: { name: string; schemaText: string }[], - suggestions: LlmSuggestion[], -): Promise { - const question = exports_.map((entry) => `${entry.name}: ${entry.schemaText}`).join("\n\n"); - try { - const answer = parseCompletionJson(await provider.complete("suggest", question, prompt)); - const proposed = answer?.tools; - if (Array.isArray(proposed)) { - for (const item of proposed) { - const entry = item as { name?: unknown; description?: unknown; write?: unknown }; - if (typeof entry.name === "string" && typeof entry.description === "string") { - suggestions.push({ - task: "suggest", - tool: entry.name, - message: - `${entry.name}: ${entry.description} ` + - `(${entry.write === true ? "write" : "read"}; declare it in codegen.config.mjs to generate)`, - }); - } - } - } - } catch (error) { - suggestions.push(failureNote("suggest", undefined, error)); - } -} - -/** A provider failure is reported as a suggestion-shaped note, never an error. */ -function failureNote(task: LlmTask, tool: string | undefined, error: unknown): LlmSuggestion { - return { - task, - tool, - message: `LLM ${task} failed (${error instanceof Error ? error.message : String(error)}). The deterministic run is unaffected.`, - }; -} diff --git a/packages/codegen/src/logger.ts b/packages/codegen/src/logger.ts index d7d6469..dc9dca2 100644 --- a/packages/codegen/src/logger.ts +++ b/packages/codegen/src/logger.ts @@ -1,7 +1,7 @@ /** * Logging via pino, pretty-printed through one in-process stream in both * TTY and CI: the report is the product's output, and a human reads CI logs - * too — piped output is the same text with the colors stripped, never JSON + * too - piped output is the same text with the colors stripped, never JSON * envelopes. (If a machine-readable mode is ever needed, it is a --json * flag, not a silent format change on pipe.) * diff --git a/packages/codegen/src/naming.test.ts b/packages/codegen/src/naming.test.ts index d3d4fad..4ff78df 100644 --- a/packages/codegen/src/naming.test.ts +++ b/packages/codegen/src/naming.test.ts @@ -92,7 +92,7 @@ describe("analyzeRoute", () => { ["patch", "/v1/users/me", "update-current-user"], ]; - it.each(cases)("%s %s → %s", (method, path, expected) => { + it.each(cases)("%s %s -> %s", (method, path, expected) => { expect(analyzeRoute(method, path).base).toBe(expected); }); diff --git a/packages/codegen/src/naming.ts b/packages/codegen/src/naming.ts index 3780783..2c76f46 100644 --- a/packages/codegen/src/naming.ts +++ b/packages/codegen/src/naming.ts @@ -6,13 +6,13 @@ * ("post-trips-trip-id-story-generate"). Every tool gets its name from the * best signal available, in order: * - * 1. An explicit override (config or .webmcp-codegen.json) — always wins. + * 1. An explicit override (config or .webmcp-codegen.json) - always wins. * 2. A cleaned operationId, when the spec has one. * 3. An intent-shaped name derived from the route (analyzeRoute). - * 4. The plain method+path concat — the total fallback that can never fail. + * 4. The plain method+path concat - the total fallback that can never fail. * * resolveNames() runs the set-level pass: collisions are deepened with - * parent context ("generate-story" → "generate-trip-story"), and every + * parent context ("generate-story" -> "generate-trip-story"), and every * rename is reported. */ @@ -25,17 +25,17 @@ export const TOOL_NAME_PATTERN = /^[a-zA-Z][a-zA-Z0-9_-]*$/; * Turn an operationId or route fragment into a valid, readable tool name. * * Examples: - * "getOrderStatus" → "get-order-status" - * "GET /orders/{id}" → "get-orders-id" - * "list_pets" → "list-pets" + * "getOrderStatus" -> "get-order-status" + * "GET /orders/{id}" -> "get-orders-id" + * "list_pets" -> "list-pets" */ export function toToolName(raw: string): string { const name = raw - // Split acronym boundaries first: "getHTTPStatus" → "get-HTTPStatus" + // Split acronym boundaries first: "getHTTPStatus" -> "get-HTTPStatus" .replace(/([A-Z]+)([A-Z][a-z])/g, "$1-$2") - // Then camelCase and PascalCase boundaries: "getOrder" → "get-Order" + // Then camelCase and PascalCase boundaries: "getOrder" -> "get-Order" .replace(/([a-z0-9])([A-Z])/g, "$1-$2") - // Path placeholders and separators become dashes: "/orders/{id}" → "-orders-id" + // Path placeholders and separators become dashes: "/orders/{id}" -> "-orders-id" .replace(/[{}/_\s.]+/g, "-") // Anything left that isn't a letter, digit or dash is dropped .replace(/[^a-zA-Z0-9-]/g, "") @@ -72,8 +72,8 @@ export function nameFromRoute(method: string, path: string): string { export function cleanOperationId(operationId: string): string { let id = operationId; - // Drop a leading framework prefix: "TripsController_create" → "create", - // "user.service.getProfile" → "getProfile". Everything up to and including + // Drop a leading framework prefix: "TripsController_create" -> "create", + // "user.service.getProfile" -> "getProfile". Everything up to and including // the first scaffolding marker goes, and only when something remains. const segments = id.split(/_|\./).filter(Boolean); const marker = segments.findIndex((s) => /(controller|service|api|handler)$/i.test(s)); @@ -81,7 +81,7 @@ export function cleanOperationId(operationId: string): string { id = segments.slice(marker + 1).join("_"); } - // Drop a trailing version marker: "createTrip_v2" → "createTrip". + // Drop a trailing version marker: "createTrip_v2" -> "createTrip". id = id.replace(/[._-]?v\d+$/i, ""); return toToolName(id); @@ -205,7 +205,7 @@ export interface RouteAnalysis { base: string; /** * "intent" names can absorb parent context on collision ("generate-story" - * → "generate-trip-story"); "fallback" names are fixed strings. + * -> "generate-trip-story"); "fallback" names are fixed strings. */ tier: "intent" | "fallback"; /** The verb and noun the name is built from: verb + context + noun. */ @@ -225,7 +225,7 @@ export interface RouteAnalysis { droppedVersion: boolean; } -/** Singularize the last word of a phrase: "saved-places" → "saved-place". */ +/** Singularize the last word of a phrase: "saved-places" -> "saved-place". */ function singularPhrase(phrase: string): string { const words = phrase.split("-"); words[words.length - 1] = pluralize.singular(words[words.length - 1] as string); @@ -260,7 +260,7 @@ function intentName( reserve: string[] = [], ): RouteAnalysis { // Context is stored nearest-first, but names read outermost-first: - // "list-explore-destination-candidates", not "list-destination-explore-…". + // "list-explore-destination-candidates", not "list-destination-explore-...". const used = context.slice(0, minDepth).reverse(); const base = [verb, ...used, noun].filter(Boolean).join("-"); return { base, tier: "intent", verb, noun, context, minDepth, droppedVersion, reserve }; @@ -271,18 +271,18 @@ function intentName( * * Two shapes are recognized: * - * Action endpoints — the last segment starts with a known verb, so the - * name is the action: POST /trips/{id}/story/generate → "generate-story". + * Action endpoints - the last segment starts with a known verb, so the + * name is the action: POST /trips/{id}/story/generate -> "generate-story". * Action names stay minimal; parent context is spent only on collision. * - * Plain REST — the method implies the verb and the path shape says member - * or collection: POST /trips → "create-trip", GET /trips/{id} → - * "get-trip", GET /trips/{id}/blocks → "list-trip-blocks". Nested reads + * Plain REST - the method implies the verb and the path shape says member + * or collection: POST /trips -> "create-trip", GET /trips/{id} -> + * "get-trip", GET /trips/{id}/blocks -> "list-trip-blocks". Nested reads * and writes include the immediate parent, because every API nests list * and get somewhere and parentless names collide constantly. * * Anything else falls back to the method+path concat, which can never fail - * to produce a name — it can only produce a boring one. + * to produce a name - it can only produce a boring one. */ export function analyzeRoute(method: string, path: string): RouteAnalysis { const segments = path.split("/").filter(Boolean); @@ -290,11 +290,11 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { const droppedVersion = rest.length !== segments.length; // Walk the path once, keeping resource segments in order and noting where - // params sit between them — a param between two resources is what makes a + // params sit between them - a param between two resources is what makes a // name nested ("trips/{id}/blocks" is nested; "trips/{id}" is not). // A member word mid-path merges into its parent as a scope marker: - // "/users/me/stamps" is read as "current-user → stamps", not "users, me, - // stamps" — "me" names no resource of its own. + // "/users/me/stamps" is read as "current-user -> stamps", not "users, me, + // stamps" - "me" names no resource of its own. const resources: { phrase: string; nestedUnderMember: boolean }[] = []; let sawParam = false; for (const segment of rest) { @@ -344,8 +344,8 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { // "batch" is a modifier, never the verb: "batch-trip-blocks" reads as a // tool about batching, not as blocks being written. The verb comes from - // the rest of the segment ("batch-delete" → "delete-media-batch") or from - // the method ("…/blocks/batch" → "update-trip-blocks-batch"). + // the rest of the segment ("batch-delete" -> "delete-media-batch") or from + // the method (".../blocks/batch" -> "update-trip-blocks-batch"). if (firstWord === "batch" && resources.length > 1) { const prev = resources[resources.length - 2] as (typeof resources)[number]; const rest = last.phrase.split("-").slice(1).join("-"); @@ -374,7 +374,7 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { // Plain REST: the method supplies the verb, the path shape the noun. const verbs = METHOD_VERBS[method.toLowerCase()]; - if (!verbs) return fallback; // HEAD, OPTIONS, WebDAV — honest concat. + if (!verbs) return fallback; // HEAD, OPTIONS, WebDAV - honest concat. // A trailing "all" scopes the parent collection rather than naming one: // "GET /pricing/all" is "list-all-pricing", never "get-all". @@ -392,9 +392,9 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { } if (lastIsParam) { - // GET /trips/{id} → the member named by the last resource segment. Two + // GET /trips/{id} -> the member named by the last resource segment. Two // or more trailing params are a lookup by the last one: - // GET /trips/{username}/{slug} → "get-trip-by-slug". + // GET /trips/{username}/{slug} -> "get-trip-by-slug". const noun = lastParam ? `${singularPhrase(last.phrase)}-by-${lastParam}` : singularPhrase(last.phrase); @@ -410,7 +410,7 @@ export function analyzeRoute(method: string, path: string): RouteAnalysis { } // A trailing resource word that reads plural is a collection; a singular - // one is a singleton sub-resource (GET /auth/session → "get-session"). + // one is a singleton sub-resource (GET /auth/session -> "get-session"). const member = !pluralize.isPlural(last.phrase.split("-").pop() as string); // POST with a trailing resource always creates one of it, whether the word // reads singular or plural: "create-trip-template", "create-trip-block". @@ -460,7 +460,7 @@ export interface ResolvedNames { * Assign final names to a whole tool set. * * Route-derived names start minimal ("generate-story") and absorb parent - * context on collision ("generate-trip-story") — minimal first because the + * context on collision ("generate-trip-story") - minimal first because the * short name is usually unique, deepening because a bare noun stops saying * which resource once a second API has one. Ties break in order: parent * context, grouping words kept in reserve ("admin" beats a number), the @@ -555,7 +555,7 @@ export function resolveNames(inputs: NameInput[]): ResolvedNames { // Nobody can deepen. The declared name (or the first arrival) keeps the // spot; the rest take a method suffix, then a counter. A method suffix // that repeats the verb says nothing ("get-trip-get"), so those go - // straight to the counter — and the counter is always an error, because + // straight to the counter - and the counter is always an error, because // a numbered name means the spec needs a human's word, not our digit. for (const group of colliding) { const ordered = [...group].sort( diff --git a/packages/codegen/src/outputs/assets.ts b/packages/codegen/src/outputs/assets.ts new file mode 100644 index 0000000..06552fd --- /dev/null +++ b/packages/codegen/src/outputs/assets.ts @@ -0,0 +1,29 @@ +/** + * Read a bundled asset (the skill file, the journey helper). + * + * These ship as files in the published package rather than as template + * strings in source because they are also the reviewable artifacts - the + * design docs and the docs site point at assets/ directly, and two sources + * of truth would drift. The package publishes dist + assets; the two + * candidate roots cover the built layout (dist/x.js -> ../assets) and the + * source tree under test (src/outputs/x.ts -> ../../assets). + */ + +import { readFile } from "node:fs/promises"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; + +export async function assetText(name: string): Promise { + const here = dirname(fileURLToPath(import.meta.url)); + const candidates = [join(here, "..", "assets", name), join(here, "..", "..", "assets", name)]; + for (const candidate of candidates) { + try { + return await readFile(candidate, "utf8"); + } catch { + // Try the next layout. + } + } + throw new Error( + `webmcp-codegen: bundled asset missing: ${name} (looked in ${candidates.join(", ")})`, + ); +} diff --git a/packages/codegen/src/outputs/generated-code.test.ts b/packages/codegen/src/outputs/generated-code.test.ts new file mode 100644 index 0000000..e53dd16 --- /dev/null +++ b/packages/codegen/src/outputs/generated-code.test.ts @@ -0,0 +1,145 @@ +import ts from "typescript"; +import { describe, expect, it } from "vitest"; +import type { ReviewedTool } from "../types.js"; +import { + barrelSource, + generatedRegion, + ownedRegionScaffold, + runtimeSource, +} from "./tools-templates.js"; + +/** + * These tests exist because the templates are TypeScript built from template + * literals, and an escape that is correct in the source file can be wrong in + * the emitted file. The 1.5K truncation notice shipped as a string literal + * broken across two lines exactly this way: the source had "\n", the outer + * template literal turned it into a real newline, and every generated + * runtime.webmcp.ts was a syntax error. Parsing the output is the only check + * that catches that class of mistake. + */ + +/** A reviewed tool with every field the templates read, overridable. */ +function reviewedTool(overrides: Partial = {}): ReviewedTool { + return { + id: "GET /orders/{id}", + name: "get-order-status", + source: { kind: "openapi", ref: "GET /orders/{id}" }, + inputSchema: { + type: "object", + properties: { orderId: { type: "string", description: "The order ID." } }, + required: ["orderId"], + }, + outputSchema: { type: "object", properties: { status: { type: "string" } } }, + inputTypeName: "GetOrderStatusInput", + httpMethod: "GET", + pathTemplate: "/orders/{orderId}", + paramLocations: { path: ["orderId"], query: [], body: [] }, + sideEffect: "read", + endpointRole: "endpoint", + requiresAuth: false, + enabledByDefault: true, + withheld: false, + description: "Returns the current status of an order.", + descriptionSource: "openapi-summary", + riskTier: "safe-read", + hints: { + readOnlyHint: true, + destructiveHint: false, + idempotentHint: true, + untrustedContentHint: false, + }, + piiInOutput: [], + ...overrides, + } as ReviewedTool; +} + +/** Syntactic diagnostics only: we do not resolve the runtime import. */ +function syntaxErrors(source: string): string[] { + const result = ts.transpileModule(source, { + reportDiagnostics: true, + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, + }); + return (result.diagnostics ?? []).map((diagnostic) => + ts.flattenDiagnosticMessageText(diagnostic.messageText, " "), + ); +} + +describe("generated files parse", () => { + it("runtime.webmcp.ts is valid syntax and keeps the truncation escape", () => { + const runtime = runtimeSource(); + expect(syntaxErrors(runtime)).toEqual([]); + // A real newline inside this double-quoted string is the bug we shipped + // once; the emitted file must carry the two-character escape instead. + expect(runtime).toContain('"\\n'); + }); + + it("an enabled endpoint tool parses", () => { + const tool = reviewedTool(); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); + + it("a withheld write tool parses, comments and all", () => { + const tool = reviewedTool({ + name: "create-trip", + id: "POST /trips", + source: { kind: "openapi", ref: "POST /trips" }, + httpMethod: "POST", + pathTemplate: "/trips", + paramLocations: { path: [], query: [], body: ["title"] }, + inputSchema: { type: "object", properties: { title: { type: "string" } } }, + sideEffect: "write", + riskTier: "write-confirm", + enabledByDefault: false, + withheld: true, + hints: { + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + untrustedContentHint: false, + }, + }); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); + + it("the barrel parses, with and without journeys", () => { + expect(syntaxErrors(barrelSource([reviewedTool()]))).toEqual([]); + expect(syntaxErrors(barrelSource([reviewedTool()], ["document-trip.webmcp.ts"]))).toEqual([]); + }); + + it("a composed handshake tool parses", () => { + const tool = reviewedTool({ + name: "upload-media", + id: "POST /media/request-upload + POST /media/uploads/{uploadId}/complete", + source: { + kind: "openapi", + ref: "POST /media/request-upload + POST /media/uploads/{uploadId}/complete", + }, + httpMethod: undefined, + pathTemplate: undefined, + paramLocations: undefined, + inputSchema: { + type: "object", + properties: { fileName: { type: "string" } }, + required: ["fileName"], + }, + sideEffect: "write", + riskTier: "write-confirm", + enabledByDefault: false, + withheld: true, + compose: { + first: { + httpMethod: "POST", + pathTemplate: "/media/request-upload", + paramLocations: { path: [], query: [], body: ["fileName"] }, + }, + second: { + httpMethod: "POST", + pathTemplate: "/media/uploads/{uploadId}/complete", + paramLocations: { path: ["uploadId"], query: [], body: [] }, + }, + threaded: { uploadId: "uploadId" }, + }, + }); + expect(syntaxErrors(`${generatedRegion(tool)}${ownedRegionScaffold(tool)}`)).toEqual([]); + }); +}); diff --git a/packages/codegen/src/outputs/tools-templates.ts b/packages/codegen/src/outputs/tools-templates.ts index 1c98e0e..7d943a0 100644 --- a/packages/codegen/src/outputs/tools-templates.ts +++ b/packages/codegen/src/outputs/tools-templates.ts @@ -33,13 +33,23 @@ import { GENERATED_END, GENERATED_START } from "./tools.js"; * track the API contract exactly: name, description, schema, input type, * hints, and the register() wrapper. */ -export function generatedRegion(tool: ReviewedTool): string { +export function generatedRegion( + tool: ReviewedTool, + registration?: { exposedTo?: string[] }, +): string { const pascal = pascalCase(tool.name); const camel = lowercaseFirst(pascal); const schemaJson = JSON.stringify(tool.inputSchema, null, 2); const inputType = jsonSchemaToTs(tool.inputSchema, undefined); const mutates = tool.riskTier !== "safe-read"; + // The second registerTool() argument: the AbortSignal always, plus the + // configured origin exposure when the app shares tools with trusted + // embedded documents (the spec's exposedTo). + const registrationOptions = registration?.exposedTo?.length + ? `{ signal, exposedTo: ${JSON.stringify(registration.exposedTo)} },` + : `{ signal },`; + // Import only what this file's regions actually use, so generated files // pass strict lint configs (no-unused-vars errors fail Next.js builds). // A disabled tool's request is commented out, so callApi/toolResult stay @@ -49,13 +59,18 @@ export function generatedRegion(tool: ReviewedTool): string { // fence, so the only runtime helper its live code uses is toolDisabled in // the execute scaffold (a defensive refusal if someone registers it by hand). const withheld = tool.withheld && !enabled; - const hasRoute = Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); + const hasRoute = + Boolean(tool.compose) || Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); const runtimeImports = withheld - ? "toolDisabled" + ? hasRoute + ? "callApi, toolDisabled" + : "toolDisabled" : [ "getModelContext", ...(mutates ? ["requestUserConfirmation"] : []), - ...(enabled && hasRoute ? ["callApi"] : []), + // Every endpoint-backed tool emits a live fetchX raw caller, so + // callApi is imported whether the tool itself starts enabled or not. + ...(hasRoute ? ["callApi"] : []), ...(enabled ? ["toolResult"] : []), "asToolError", ...(enabled ? [] : ["toolDisabled"]), @@ -89,7 +104,7 @@ export function generatedRegion(tool: ReviewedTool): string { ` }`, ` },`, ` },`, - ` { signal },`, + ` ${registrationOptions}`, ` );`, ] : [ @@ -106,7 +121,7 @@ export function generatedRegion(tool: ReviewedTool): string { ` }`, ` },`, ` },`, - ` { signal },`, + ` ${registrationOptions}`, ` );`, ]; @@ -141,13 +156,27 @@ export function generatedRegion(tool: ReviewedTool): string { `/** The tool definition, minus \`execute\` (which is yours, below the marker). */`, `export const ${camel}Tool = {`, ` name: ${JSON.stringify(tool.name)},`, + ` title: ${JSON.stringify(titleFromName(tool.name))},`, ` description: ${JSON.stringify(tool.description)},`, ` inputSchema: ${camel}InputSchema,`, ` annotations: {`, ` readOnlyHint: ${tool.hints.readOnlyHint},`, ` untrustedContentHint: ${tool.hints.untrustedContentHint},`, + ` consequentialHint: ${tool.riskTier === "destructive-confirm"},`, ` },`, `};`, + ...(hasRoute + ? [ + ``, + `/** The bare request, without the agent-facing result wrapping. Journeys`, + ` * and your own code compose this; execute${pascal} is the agent-facing one. */`, + `export async function fetch${pascal}(input: ${tool.inputTypeName}, signal?: AbortSignal) {`, + ...(tool.compose + ? composedFetchBody(tool.compose) + : [` ${requestCall(tool)}`, ` return data;`]), + `}`, + ] + : []), ``, ...(withheld ? [ @@ -199,7 +228,12 @@ export function generatedRegion(tool: ReviewedTool): string { */ export function ownedRegionScaffold(tool: ReviewedTool): string { const pascal = pascalCase(tool.name); - const call = requestCall(tool); + const hasRoute = + Boolean(tool.compose) || Boolean(tool.httpMethod && tool.pathTemplate && tool.paramLocations); + // Endpoint-backed tools compose the raw caller from the generated region + // (fetchX), so the default execute stays one line and journeys reuse the + // exact same request. Schema-only tools keep the honest TODO. + const call = hasRoute ? `const data = await fetch${pascal}(input, signal);` : requestCall(tool); // An endpoint-backed tool scaffolds a call to its route. A standalone schema // tool has no route: the honest scaffold says "wire this to your app's own // action" and names nothing we made up. @@ -247,7 +281,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { if (tool.piiInOutput.length > 0) { lines.push( `//`, - `// ⚠ webmcp-codegen flagged these response fields as likely PII: ${tool.piiInOutput.join(", ")}.`, + `// ! webmcp-codegen flagged these response fields as likely PII: ${tool.piiInOutput.join(", ")}.`, `// Everything you return reaches the agent. Leave those fields out of what you`, `// return unless the agent genuinely needs them, and say so in a comment if you keep them.`, ); @@ -276,7 +310,10 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { ? [ ` // This tool is withheld: nothing registers it, so agents cannot see`, ` // or call it. To enable it, uncomment the request below and the`, - ` // registration above, and add callApi and toolResult to the import.`, + // Route-backed tools import callApi already (fetchX uses it). + hasRoute + ? ` // registration above, and add toolResult to the import.` + : ` // registration above, and add callApi and toolResult to the import.`, ] : [ ` // This tool starts disabled: it ${ @@ -285,7 +322,9 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { : `wraps an ${tool.endpointRole} endpoint` }. Agents can see it, and calling it tells`, ` // them it is disabled. To enable it, delete the line below, uncomment`, - ` // the code, and add callApi and toolResult to the import above.`, + hasRoute + ? ` // the code, and add toolResult to the import above.` + : ` // the code, and add callApi and toolResult to the import above.`, ]), ` void signal; // passed to fetch once you enable the call below`, ` return toolDisabled("${tool.name}.webmcp.ts");`, @@ -304,7 +343,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { * knows: the path template becomes a template literal, query params become * the search string, body fields become the JSON body. * - * "/pets/{id}" + DELETE → const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); + * "/pets/{id}" + DELETE -> const data = await callApi(`/pets/${input.id}`, { method: "DELETE" }); * * When the source carries no route information, we fall back to an honest * TODO instead of inventing a URL. @@ -314,7 +353,7 @@ export function ownedRegionScaffold(tool: ReviewedTool): string { * is frequently a local dev URL (http://localhost:3001); baking that into the * generated fetch makes every deployed tool call the visitor's own machine. * So: a non-local absolute URL is kept (the API genuinely lives elsewhere), - * a local one returns undefined so the tool falls back to same-origin — + * a local one returns undefined so the tool falls back to same-origin - * which is where a deployed app's API actually is. */ export function resolveApiBase(serverUrl: string | undefined): string | undefined { @@ -330,46 +369,103 @@ export function resolveApiBase(serverUrl: string | undefined): string | undefine } } -function requestCall(tool: ReviewedTool): string { - if (!tool.httpMethod || !tool.pathTemplate || !tool.paramLocations) { - return `const data = null; // TODO: call your app's existing code here.`; - } - - const { path: pathParams, query: queryParams, body: bodyParams } = tool.paramLocations; - - // "/pets/{id}" → `/pets/${input.id}`. Params the schema knows by name. - let pathExpr = `\`${tool.pathTemplate.replace(/\{([^}]+)\}/g, (_m, param: string) => `\${${inputRef(param)}}`)}\``; - if (pathParams.length === 0) pathExpr = JSON.stringify(tool.pathTemplate); +/** + * The arguments to one callApi(...): the path expression (template params + * interpolated, non-local server URLs kept) plus method/query/body/signal. + * `pathRef` says where each path param's value comes from - ordinary tools + * read `input`, the second call of a composed tool reads the first result. + */ +function buildCallExpr(options: { + httpMethod: string; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + pathRef?: (param: string) => string; + skipFields?: Set; +}): string { + const { + httpMethod, + pathTemplate, + paramLocations, + serverUrl, + pathRef = inputRef, + skipFields = new Set(), + } = options; + const { path: pathParams, query: queryParamsAll, body: bodyParamsAll } = paramLocations; + const queryParams = queryParamsAll.filter((name) => !skipFields.has(name)); + const bodyParams = bodyParamsAll.filter((name) => !skipFields.has(name)); + + // "/pets/{id}" -> `/pets/${input.id}`. Params the schema knows by name. + let pathExpr = `\`${pathTemplate.replace(/\{([^}]+)\}/g, (_m, param: string) => `\${${pathRef(param)}}`)}\``; + if (pathParams.length === 0) pathExpr = JSON.stringify(pathTemplate); // Base URL: default to the page's own origin so a deployed app calls its // own API. Baking the spec's servers[0] (often http://localhost:3001) into // the generated fetch would make every deployed tool call the visitor's // own machine. A non-local public server URL is kept as the default base; // a local one is not. - const base = resolveApiBase(tool.serverUrl); + const base = resolveApiBase(serverUrl); if (base) { const b = base.endsWith("/") ? base.slice(0, -1) : base; pathExpr = `\`${b}\${${pathExpr}}\``; } - const options: string[] = [`method: ${JSON.stringify(tool.httpMethod)}`]; + const args: string[] = [`method: ${JSON.stringify(httpMethod)}`]; if (queryParams.length > 0) { const entries = queryParams.map((name) => `${safeKey(name)}: ${inputRef(name)}`).join(", "); - options.push(`query: { ${entries} }`); + args.push(`query: { ${entries} }`); } if (bodyParams.length > 0) { if (bodyParams.length === 1 && bodyParams[0] === "body") { // A non-object request body arrives as a single "body" field. - options.push(`body: input.body`); + args.push(`body: input.body`); } else { const entries = bodyParams.map((name) => `${safeKey(name)}: ${inputRef(name)}`).join(", "); - options.push(`body: { ${entries} }`); + args.push(`body: { ${entries} }`); } } // The execute context's signal reaches fetch, so a cancelled call stops. - options.push("signal"); + args.push("signal"); - return `const data = await callApi(${pathExpr}, { ${options.join(", ")} });`; + return `${pathExpr}, { ${args.join(", ")} }`; +} + +function requestCall(tool: ReviewedTool): string { + if (!tool.httpMethod || !tool.pathTemplate || !tool.paramLocations) { + return `const data = null; // TODO: call your app's existing code here.`; + } + return `const data = await callApi(${buildCallExpr({ + httpMethod: tool.httpMethod, + pathTemplate: tool.pathTemplate, + paramLocations: tool.paramLocations, + serverUrl: tool.serverUrl, + })});`; +} + +/** "uploadId" on the first response: dot access when the name allows it. */ +function firstResultRef(param: string): string { + return /^[a-zA-Z_$][a-zA-Z0-9_$]*$/.test(param) + ? `firstResult.${param}` + : `firstResult[${JSON.stringify(param)}]`; +} + +/** + * The two calls of a grouped handshake tool: the first request runs, its + * response fields fill the second request's path params by exact name, and + * the second response is the tool's result. Threaded fields never reach the + * agent-facing input - that's the point of the composition. + */ +function composedFetchBody(plan: NonNullable): string[] { + const threaded = new Set(Object.keys(plan.threaded)); + return [ + ` const firstResult = (await callApi(${buildCallExpr(plan.first)})) as Record;`, + ` const data = await callApi(${buildCallExpr({ + ...plan.second, + pathRef: (param) => (threaded.has(param) ? firstResultRef(param) : inputRef(param)), + skipFields: threaded, + })});`, + ` return data;`, + ]; } /** @@ -408,10 +504,16 @@ export interface WebMcpToolResult { /** A tool as the browser runtime understands it. */ export interface WebMcpToolDefinition { name: string; + /** A human-facing label for native UIs (the spec's USVString title). */ + title?: string; description: string; inputSchema?: Record; /** Hints the agent reads to decide how careful to be with this tool. */ - annotations?: { readOnlyHint?: boolean; untrustedContentHint?: boolean }; + annotations?: { + readOnlyHint?: boolean; + untrustedContentHint?: boolean; + consequentialHint?: boolean; + }; execute: ( input: Record, context?: { signal?: AbortSignal }, @@ -422,7 +524,7 @@ export interface WebMcpToolDefinition { export interface ModelContext { registerTool( tool: WebMcpToolDefinition, - options?: { signal?: AbortSignal }, + options?: { signal?: AbortSignal; exposedTo?: string[] }, ): Promise; } @@ -447,6 +549,30 @@ export function getModelContext(): ModelContext | null { return modelContext ?? null; } +/** + * Register every journey exported from the modules the barrel found in + * journeys/. Anything with a .register() method counts (createJourney's + * return shape); anything else is skipped quietly. One journey failing never + * takes the others down with it. + */ +export async function registerJourneys( + modules: Record[], + signal?: AbortSignal, +): Promise { + for (const module of modules) { + for (const value of Object.values(module)) { + const journey = value as { register?: unknown } | null; + if (journey !== null && typeof journey === "object" && typeof journey.register === "function") { + try { + await (journey.register as (signal?: AbortSignal) => Promise)(signal); + } catch (error) { + console.warn("[webmcp-codegen] a journey failed to register:", error); + } + } + } + } +} + /** * Call your API from the page. Same origin by default (pass a full URL when * the API lives on another host), always with the signed-in user's session @@ -486,12 +612,27 @@ export async function callApi( } } -/** Wrap a result in the MCP shape, so tool bodies stay one line. */ +/** Chrome's output budget: one tool result stays under ~1.5K characters. */ +const TOOL_OUTPUT_MAX = 1536; + +const TRUNCATED_NOTICE = + "\\n... [truncated to fit the 1.5K output budget - return a smaller slice or paginate]"; + +/** + * Wrap a result in the MCP shape, so tool bodies stay one line. The result + * text is capped at Chrome's ~1.5K per-call output budget: oversized payloads + * cost the agent context and can trip guardrails, so they are cut with a + * notice rather than delivered whole. The cap lives here in the shared + * runtime, so it cannot be edited away per tool. + */ export function toolResult(data: unknown): WebMcpToolResult { + const text = typeof data === "string" ? data : JSON.stringify(data, null, 2); + const fitted = + text.length <= TOOL_OUTPUT_MAX + ? text + : text.slice(0, TOOL_OUTPUT_MAX - TRUNCATED_NOTICE.length) + TRUNCATED_NOTICE; return { - content: [ - { type: "text", text: typeof data === "string" ? data : JSON.stringify(data, null, 2) }, - ], + content: [{ type: "text", text: fitted }], }; } @@ -542,12 +683,20 @@ export function requestUserConfirmation(message: string): Promise { } /** The barrel: one import that registers every generated tool. */ -export function barrelSource(tools: ReviewedTool[]): string { +export function barrelSource(tools: ReviewedTool[], journeyFiles: string[] = []): string { const imports = tools .map((tool) => `import { register${pascalCase(tool.name)} } from "./${tool.name}.webmcp";`) .join("\n"); const names = tools.map((tool) => `register${pascalCase(tool.name)}`).join(",\n "); + const journeyImports = journeyFiles + .map( + (file, index) => + `import * as journeyModule${index} from "./journeys/${file.replace(/\.ts$/, "")}";`, + ) + .join("\n"); + const journeyModuleNames = journeyFiles.map((_, index) => `journeyModule${index}`).join(", "); + return `/** * Generated by webmcp-codegen. This file is fully regenerated on every run. * Import registerAllTools() once at app startup: @@ -557,11 +706,11 @@ export function barrelSource(tools: ReviewedTool[]): string { */ ${imports} - +${journeyFiles.length > 0 ? `\nimport { registerJourneys } from "./runtime.webmcp";\n${journeyImports}\n` : ""} const registrations = [ ${names} ]; - +${journeyFiles.length > 0 ? `\nconst journeyModules = [${journeyModuleNames}];\n` : ""} /** * Register every generated tool with WebMCP. One tool failing (for example * because the page's Permissions-Policy disables tools) never takes the @@ -575,11 +724,30 @@ export async function registerAllTools(signal?: AbortSignal): Promise { console.warn("[webmcp-codegen] a tool failed to register:", error); } } + ${ + journeyFiles.length > 0 + ? `// Journeys come last: their steps compose the tools above. + await registerJourneys(journeyModules, signal);` + : `// Drop journey definitions into ./journeys/ and re-run \`generate\`: + // the next barrel registers every createJourney() export it finds there.` + } } `; } -/** "GetOrderStatus" → "getOrderStatus" (for the generated const names). */ +/** "GetOrderStatus" -> "getOrderStatus" (for the generated const names). */ +/** + * The spec's human-facing `title`: "list-trips" -> "List Trips". Derived from + * the name so the two never disagree. + */ +function titleFromName(name: string): string { + return name + .split("-") + .filter(Boolean) + .map((part) => part.charAt(0).toUpperCase() + part.slice(1)) + .join(" "); +} + function lowercaseFirst(pascal: string): string { return pascal.charAt(0).toLowerCase() + pascal.slice(1); } diff --git a/packages/codegen/src/outputs/tools.test.ts b/packages/codegen/src/outputs/tools.test.ts index f52ba17..a37aca2 100644 --- a/packages/codegen/src/outputs/tools.test.ts +++ b/packages/codegen/src/outputs/tools.test.ts @@ -58,6 +58,71 @@ describe("js generator", () => { await rm(cwd, { recursive: true, force: true }); }); + it("emits a live raw caller (fetchX) that the default execute composes", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const tool = files.find((file) => file.path.includes("get-order-status")); + expect(tool?.contents).toContain("export async function fetchGetOrderStatus("); + expect(tool?.contents).toContain("const data = await fetchGetOrderStatus(input, signal);"); + }); + + it("scaffolds the journey factory and never reports it as an orphan", async () => { + // Simulate a previous run's factory sitting on disk. + await mkdir(join(cwd, "src/webmcp"), { recursive: true }); + await writeFile(join(cwd, "src/webmcp/journey.webmcp.ts"), "// previously scaffolded\n"); + + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const helper = files.find((file) => file.path.endsWith("webmcp/journey.webmcp.ts")); + expect(helper).toBeDefined(); + expect(helper?.contents).toContain("export function createJourney("); + + // Generator-owned, so the orphan report must leave it alone. + const barrel = files.find((file) => file.path.endsWith("webmcp/index.ts")); + expect((barrel?.notes ?? []).join("\n")).not.toContain("journey.webmcp.ts"); + }); + + it("registers journey files found in the journeys folder", async () => { + await mkdir(join(cwd, "src/webmcp/journeys"), { recursive: true }); + await writeFile( + join(cwd, "src/webmcp/journeys/document-trip.webmcp.ts"), + "// the user's journey definition\n", + ); + + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const barrel = files.find((file) => file.path.endsWith("webmcp/index.ts")); + expect(barrel?.contents).toContain( + 'import * as journeyModule0 from "./journeys/document-trip.webmcp";', + ); + expect(barrel?.contents).toContain("await registerJourneys(journeyModules, signal);"); + const runtime = files.find((file) => file.path.endsWith("webmcp/runtime.webmcp.ts")); + expect(runtime?.contents).toContain("export async function registerJourneys("); + }); + + it("scaffolds the WebMCP skill file at the cross-client skills location", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const skill = files.find((file) => file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md")); + expect(skill).toBeDefined(); + expect(skill?.action).toBe("create"); + expect(skill?.contents).toContain("name: webmcp-tools"); + expect(skill?.contents).toContain("Regenerated by webmcp-codegen"); + + // A copy already on disk matching the asset reports unchanged. + const skillPath = join(cwd, ".agents/skills/webmcp-tools/SKILL.md"); + await mkdir(dirname(skillPath), { recursive: true }); + await writeFile(skillPath, skill?.contents ?? ""); + const again = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const skillAgain = again.find((file) => + file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md"), + ); + expect(skillAgain?.action).toBe("unchanged"); + }); + + it("caps tool results at the 1.5K output budget in the shared runtime", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); + const runtime = files.find((file) => file.path.endsWith("runtime.webmcp.ts")); + expect(runtime?.contents).toContain("TOOL_OUTPUT_MAX"); + expect(runtime?.contents).toContain("truncated to fit the 1.5K output budget"); + }); + it("emits a runtime, a barrel, and one file per tool", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const paths = files.map((file) => file.path); @@ -141,8 +206,11 @@ describe("js generator", () => { // Commented like an editor's toggle-comment: fixed marker column, original // indentation preserved, so uncommenting restores working code. expect(tool?.contents).toContain("// ...cancelOrderTool,"); - // The import reflects it: getModelContext is part of the fence, not the file. - expect(tool?.contents).toContain('import { toolDisabled } from "./runtime.webmcp";'); + // The import reflects it: getModelContext is part of the fence, not the + // file - but callApi stays live, because the raw caller (fetchX) the + // generated region emits is live too: journeys compose withheld tools. + expect(tool?.contents).toContain('import { callApi, toolDisabled } from "./runtime.webmcp";'); + expect(tool?.contents).toContain("export async function fetchCancelOrder("); // The execute scaffold still refuses politely if someone registers it by hand. expect(tool?.contents).toContain('return toolDisabled("cancel-order.webmcp.ts");'); // And the header says what "withheld" means. @@ -175,6 +243,38 @@ describe("js generator", () => { expect(tool?.contents).toContain("untrustedContentHint: true"); }); + it("emits a human-facing title and marks destructive tools consequential", async () => { + const files = await tools({ outDir: "src/webmcp" }).generate( + [ + reviewedTool(), + reviewedTool({ + name: "delete-order", + httpMethod: "DELETE", + riskTier: "destructive-confirm", + sideEffect: "write", + enabledByDefault: false, + withheld: true, + }), + ], + cwd, + ); + const read = files.find((file) => file.path.includes("get-order-status")); + const destructive = files.find((file) => file.path.includes("delete-order")); + expect(read?.contents).toContain('title: "Get Order Status"'); + expect(read?.contents).toContain("consequentialHint: false"); + expect(destructive?.contents).toContain('title: "Delete Order"'); + expect(destructive?.contents).toContain("consequentialHint: true"); + }); + + it("passes configured exposedTo origins to registerTool", async () => { + const files = await tools({ + outDir: "src/webmcp", + exposedTo: ["https://partner.example"], + }).generate([reviewedTool()], cwd); + const tool = files.find((file) => file.path.includes("get-order-status")); + expect(tool?.contents).toContain('{ signal, exposedTo: ["https://partner.example"] }'); + }); + it("registration skips quietly when the browser has no WebMCP", async () => { const files = await tools({ outDir: "src/webmcp" }).generate([reviewedTool()], cwd); const tool = files.find((file) => file.path.includes("get-order-status")); @@ -232,12 +332,11 @@ describe("js generator", () => { cwd, ); const tool = files.find((file) => file.path.includes("cancel-order")); - // Disabled notice first, the working call right below it, commented out. + // Disabled notice first, the working call right below it, commented out + // - and it composes the generated region's raw caller like every other + // endpoint-backed tool. expect(tool?.contents).toContain('return toolDisabled("cancel-order.webmcp.ts");'); - expect(tool?.contents).toContain( - // biome-ignore lint/suspicious/noTemplateCurlyInString: asserting on generated source, which contains a template literal - "// const data = await callApi(`/orders/${input.orderId}/cancel`", - ); + expect(tool?.contents).toContain("// const data = await fetchCancelOrder(input, signal);"); // The consent gate is generated, not left as a comment for humans to remember. expect(tool?.contents).toContain("requestUserConfirmation("); expect(tool?.contents).toContain("The user declined this action."); @@ -277,13 +376,14 @@ describe("js generator", () => { 'import { getModelContext, callApi, toolResult, asToolError } from "./runtime.webmcp";', ); - // The disabled tool's request is commented out, so its helpers stay out - // of the import line; the enable instructions name what to add back. + // The disabled tool's request is commented out, but its generated raw + // caller (fetchX) is live - so callApi is in the import line, and the + // enable instructions only name what's genuinely missing. const disabledWrite = files.find((file) => file.path.includes("cancel-order")); expect(disabledWrite?.contents).toContain( - 'import { getModelContext, requestUserConfirmation, asToolError, toolDisabled } from "./runtime.webmcp";', + 'import { getModelContext, requestUserConfirmation, callApi, asToolError, toolDisabled } from "./runtime.webmcp";', ); - expect(disabledWrite?.contents).toContain("add callApi and toolResult to the import above"); + expect(disabledWrite?.contents).toContain("add toolResult to the import above"); // A standalone schema tool has no route to call: no callApi. It is a // write, so the confirmation gate's helper is imported and used. @@ -301,6 +401,9 @@ describe("js generator", () => { const files = await tools({ outDir: absoluteOut }).generate([reviewedTool()], cwd); expect(files.length).toBeGreaterThan(0); for (const file of files) { + // The skill file is the one exception: it lives at the project root's + // cross-client skills location, not under outDir. + if (file.path.endsWith(".agents/skills/webmcp-tools/SKILL.md")) continue; expect(file.path.startsWith(absoluteOut)).toBe(true); expect(file.path.startsWith(cwd)).toBe(false); } @@ -432,7 +535,7 @@ describe("js generator", () => { await writeAll( await output.generate( [ - reviewedTool(), // get-order-status → GET /orders/{id} + reviewedTool(), // get-order-status -> GET /orders/{id} reviewedTool({ id: "DELETE /admin/orders/{id}", name: "delete-order", diff --git a/packages/codegen/src/outputs/tools.ts b/packages/codegen/src/outputs/tools.ts index 69a3d8d..76de72d 100644 --- a/packages/codegen/src/outputs/tools.ts +++ b/packages/codegen/src/outputs/tools.ts @@ -6,15 +6,15 @@ * Output layout for `tools({ outDir: "./src/webmcp" })`: * * src/webmcp/ - * ├── runtime.webmcp.ts ← fully generated, never edit - * ├── index.ts ← fully generated, registers everything - * ├── get-order-status.webmcp.ts ← generated contract + YOUR execute() - * └── ... + * |-- runtime.webmcp.ts <- fully generated, never edit + * |-- index.ts <- fully generated, registers everything + * |-- get-order-status.webmcp.ts <- generated contract + YOUR execute() + * `-- ... * * Each per-tool file has two regions, divided by marker comments: * * generated region schema, input type, tool definition, register() - * ── end generated ── everything below survives regeneration + * -- end generated -- everything below survives regeneration * your region execute(), scaffolded once, then owned by you * * A tool's identity is its source ref (the endpoint, or the declared schema @@ -32,6 +32,7 @@ import { readdir, readFile } from "node:fs/promises"; import { basename, join, resolve } from "node:path"; import type { GeneratedFile, Output, ReviewedTool } from "../types.js"; +import { assetText } from "./assets.js"; import { barrelSource, generatedRegion, @@ -42,6 +43,13 @@ import { export interface ToolsOutputOptions { /** Where the tool files go, relative to the project root. */ outDir: string; + /** + * The spec's `exposedTo`: secure origins (embedded documents at these + * origins) the registered tools are shared with. Absent means the default - + * tools are visible to the page itself, same-origin documents, and the + * browser's built-in agent. Only list origins you trust to act for your user. + */ + exposedTo?: string[]; } /** @@ -50,9 +58,18 @@ export interface ToolsOutputOptions { * and we must never touch anything after it. tools-templates.ts imports * these so the marker text is defined in exactly one place. */ -export const GENERATED_START = "// ─── webmcp-codegen: generated. Do not edit this region. ───"; +export const GENERATED_START = "// --- webmcp-codegen: generated. Do not edit this region. ---"; export const GENERATED_END = - "// ─── webmcp-codegen: end generated. Your code below survives regeneration. ───"; + "// --- webmcp-codegen: end generated. Your code below survives regeneration. ---"; + +/** + * The pre-0.9 end marker, written with box-drawing characters. Reads still + * accept it so files generated by an older version migrate to the plain + * marker on the next run instead of looking like a hand edit. Written with + * escapes so this source file stays plain text. + */ +const GENERATED_END_LEGACY = + "// \u2500\u2500\u2500 webmcp-codegen: end generated. Your code below survives regeneration. \u2500\u2500\u2500"; /** Create the `tools` output for the config's `outputs` array. */ export function tools(options: ToolsOutputOptions): Output { @@ -71,6 +88,9 @@ export function tools(options: ToolsOutputOptions): Output { try { for (const entry of await readdir(outDir)) { if (!entry.endsWith(".webmcp.ts") || entry === "runtime.webmcp.ts") continue; + // The journey factory is generator-owned (regenerated wholesale + // below), never an orphan candidate. + if (entry === "journey.webmcp.ts") continue; const path = join(outDir, entry); existing.set(path, await readFile(path, "utf8")); } @@ -78,7 +98,19 @@ export function tools(options: ToolsOutputOptions): Output { // First run: the directory does not exist yet. } - // Endpoint ref → the file currently holding it. + // Journey definitions the user (or their agent) wrote since the last + // run. The barrel imports and registers them; dropping a new file in + // here and re-running generate is the whole wiring story. + let journeyFiles: string[] = []; + try { + journeyFiles = (await readdir(join(outDir, "journeys"))) + .filter((entry) => entry.endsWith(".webmcp.ts")) + .sort(); + } catch { + // No journeys directory yet - most repos, most of the time. + } + + // Endpoint ref -> the file currently holding it. const pathByRef = new Map(); for (const [path, contents] of existing) { const ref = sourceRefOf(contents); @@ -119,6 +151,7 @@ export function tools(options: ToolsOutputOptions): Output { existing.get(sourcePath), atOwnPath !== undefined && sourcePath !== ownPath, notes, + options.exposedTo, ), ); } else if (atOwnPath !== undefined) { @@ -134,10 +167,10 @@ export function tools(options: ToolsOutputOptions): Output { ]; files.push(aside); consumedPaths.add(ownPath); - files.push(toolFile(tool, ownPath, undefined, true, notes)); + files.push(toolFile(tool, ownPath, undefined, true, notes, options.exposedTo)); } else { // Brand new tool: lay down the execute() scaffold with it. - files.push(toolFile(tool, ownPath, undefined, false, notes)); + files.push(toolFile(tool, ownPath, undefined, false, notes, options.exposedTo)); } } @@ -155,9 +188,32 @@ export function tools(options: ToolsOutputOptions): Output { // The runtime and the barrel are regenerated wholesale every run; // their headers say "do not edit", and we mean it. Orphan reports ride // on the barrel so they surface even when nothing else changed. - const barrel = await plainFile(join(outDir, "index.ts"), barrelSource(tools)); + const barrel = await plainFile(join(outDir, "index.ts"), barrelSource(tools, journeyFiles)); barrel.notes = [...orphanNotes, ...(barrel.notes ?? [])]; files.unshift(await plainFile(join(outDir, "runtime.webmcp.ts"), runtimeSource()), barrel); + + // The journey factory: fully ours, regenerated wholesale like the + // runtime. Journey definitions (the user's code) import createJourney + // from it. + files.push( + await plainFile(join(outDir, "journey.webmcp.ts"), await assetText("journey.webmcp.ts")), + ); + + // The skill file: the rules harness for the user's own coding agents, + // at the cross-client skills location. Regenerated wholesale like the + // runtime - its header comment says why (project rules belong in the + // user's own skill directory, which stacks on top). + const skillFile = await plainFile( + resolve(cwd, ".agents/skills/webmcp-tools/SKILL.md"), + await assetText("skill/SKILL.md"), + ); + if (skillFile.action !== "unchanged") { + skillFile.notes = [ + "The WebMCP skill for coding agents lives at .agents/skills/webmcp-tools/SKILL.md.", + ...(skillFile.notes ?? []), + ]; + } + files.push(skillFile); return files; }, }; @@ -194,8 +250,9 @@ function toolFile( existing: string | undefined, targetOccupied: boolean, notes: string[], + exposedTo?: string[], ): GeneratedFile { - const head = generatedRegion(tool); + const head = generatedRegion(tool, { exposedTo }); if (existing === undefined) { return { @@ -206,8 +263,12 @@ function toolFile( }; } - const markerIndex = existing.indexOf(GENERATED_END); - if (markerIndex === -1) { + const endMarker = existing.includes(GENERATED_END) + ? GENERATED_END + : existing.includes(GENERATED_END_LEGACY) + ? GENERATED_END_LEGACY + : undefined; + if (endMarker === undefined) { // Someone removed the markers or hand-wrote this path from scratch. // Never clobber their work: report a conflict and let the pipeline put // our version in a `.new` sibling for a human to merge. @@ -215,7 +276,7 @@ function toolFile( } // Keep everything the developer wrote below the marker, word for word. - const preservedTail = existing.slice(markerIndex + GENERATED_END.length); + const preservedTail = existing.slice(existing.indexOf(endMarker) + endMarker.length); const contents = head + preservedTail; return { path, diff --git a/packages/codegen/src/pipeline.ts b/packages/codegen/src/pipeline.ts index d75b888..3f25533 100644 --- a/packages/codegen/src/pipeline.ts +++ b/packages/codegen/src/pipeline.ts @@ -1,5 +1,5 @@ /** - * The pipeline: sources → normalize → safety review → audit → write. + * The pipeline: sources -> normalize -> safety review -> audit -> write. * * This module is the only place the stages meet. It owns no opinions of its * own; naming, safety, and file formats all live in their own modules. It @@ -10,8 +10,8 @@ import { mkdir, readFile, writeFile } from "node:fs/promises"; import { dirname, resolve } from "node:path"; import { describeCandidateInputs, describeCandidateTool } from "./describe.js"; +import { groupHandshakes } from "./group.js"; import { pascalCase } from "./json-schema.js"; -import { runLlmLayer } from "./llm.js"; import { mergeSchemaWithOperations } from "./merge.js"; import { resolveNames } from "./naming.js"; import { auditTools, reviewTools } from "./safety.js"; @@ -19,7 +19,6 @@ import type { AuditFinding, CodegenConfig, GeneratedFile, - LlmSuggestion, ReviewedTool, SkippedEndpoint, ToolOverrides, @@ -41,7 +40,7 @@ export interface GenerateOptions { */ overrides?: ToolOverrides; /** - * The names the last run produced (name → route ref), from the same file. + * The names the last run produced (name -> route ref), from the same file. * When a route's name changes between runs, the rename is reported and the * tool's overrides move with it: a rename is a report line, never a silent * break. @@ -59,14 +58,9 @@ export interface GenerateResult { files: GeneratedFile[]; /** Human-facing pipeline notes, e.g. "stripped the shared v1 prefix". */ notes: string[]; - /** - * Advisory proposals from the LLM layer (`◦` lines in the report). Empty - * unless the layer is explicitly configured; never applied to files. - */ - suggestions: LlmSuggestion[]; - /** Names that changed since the last run (old → new), overrides re-keyed. */ + /** Names that changed since the last run (old -> new), overrides re-keyed. */ crossRenames: { from: string; to: string }[]; - /** The names this run produced (name → route ref), for the caller to save. */ + /** The names this run produced (name -> route ref), for the caller to save. */ namesLedger: Record; /** Overrides with renamed tools re-keyed, when a rename moved any. */ migratedOverrides?: ToolOverrides; @@ -131,15 +125,23 @@ export async function runGenerate( progress(`Renamed ${renames.length} tool${renames.length === 1 ? "" : "s"} for uniqueness`); } + // 4.5 Grouping: handshake endpoints (request-upload + complete-upload) are + // one action the API split into two calls. The merged tool is a + // withheld draft exactly like its members; the report names the + // proposal, and adopting it is enabling it. Members stay untouched. + const grouped = groupHandshakes(named); + notes.push(...grouped.notes); + const groupedTools = grouped.tools; + // Cross-run renames. The route ref is the durable identity; the name is // derived. When they drift apart (a better algorithm, a spec edit), the // tool's dashboard overrides are keyed by the old name and would silently - // stop applying — so they are re-keyed here, before step 6 reads them. + // stop applying - so they are re-keyed here, before step 6 reads them. const refOf = (tool: (typeof named)[number]): string => tool.endpointRef ?? tool.source.ref; const crossRenames: { from: string; to: string }[] = []; if (options.previousNames) { const nameByRef = new Map(Object.entries(options.previousNames).map(([n, r]) => [r, n])); - for (const tool of named) { + for (const tool of groupedTools) { const before = nameByRef.get(refOf(tool)); if (before && before !== tool.name) crossRenames.push({ from: before, to: tool.name }); } @@ -159,12 +161,12 @@ export async function runGenerate( `${crossRenames.length} tool${crossRenames.length === 1 ? "" : "s"} renamed since the last run; their dashboard edits moved with them`, ); } - const namesLedger = Object.fromEntries(named.map((tool) => [tool.name, refOf(tool)])); + const namesLedger = Object.fromEntries(groupedTools.map((tool) => [tool.name, refOf(tool)])); // 5. Safety review: classify side effects, compute hints, scan for PII, // apply endpoint roles and config exclusions. Webhooks never come back. progress("Reviewing safety (classification, PII, auth)"); - const { tools, skipped } = reviewTools(named, config.safety); + const { tools, skipped } = reviewTools(groupedTools, config.safety); const authCount = tools.filter((t) => t.endpointRole === "auth").length; const adminCount = tools.filter((t) => t.endpointRole === "admin").length; if (authCount > 0) progress(`Disabled ${authCount} auth endpoint${authCount === 1 ? "" : "s"}`); @@ -233,7 +235,7 @@ export async function runGenerate( level: "warning" as const, tool: rename.to, message: - `Renamed "${rename.from}" → "${rename.to}" since the last run. ` + + `Renamed "${rename.from}" -> "${rename.to}" since the last run. ` + "Dashboard edits moved with it; update any code that imported the old name.", })), ...formFindings, @@ -255,7 +257,6 @@ export async function runGenerate( findings, files: [], notes, - suggestions: [], crossRenames, namesLedger, migratedOverrides, @@ -264,12 +265,6 @@ export async function runGenerate( }; } - // 7b. The advisory LLM layer runs after the audit so its relationship - // proposals can react to findings, and before outputs so a slow endpoint - // never sits between the developer and their files. It only proposes: - // report lines, never writes, never exit codes. - const suggestions = await runLlmLayer(config, { tools, findings }); - // 8. Run the outputs, then write the files (unless this is a dry run). // Tools with a form pointer belong to the form output; without one // configured they generate as ordinary tool files, loudly. @@ -316,7 +311,6 @@ export async function runGenerate( findings, files, notes, - suggestions, crossRenames, namesLedger, migratedOverrides, diff --git a/packages/codegen/src/safety.ts b/packages/codegen/src/safety.ts index 077eb41..9b5a9a9 100644 --- a/packages/codegen/src/safety.ts +++ b/packages/codegen/src/safety.ts @@ -310,7 +310,7 @@ export function auditTools( findings.push({ level: "warning", tool: rename.to, - message: `Renamed "${rename.from}" → "${rename.to}" to keep tool names unique.`, + message: `Renamed "${rename.from}" -> "${rename.to}" to keep tool names unique.`, }); } diff --git a/packages/codegen/src/sources/openapi.test.ts b/packages/codegen/src/sources/openapi.test.ts index 8473f5b..5929bae 100644 --- a/packages/codegen/src/sources/openapi.test.ts +++ b/packages/codegen/src/sources/openapi.test.ts @@ -111,7 +111,7 @@ describe("openapi source", () => { expect(tools.map((tool) => tool.name)).toEqual([ "list-orders", "create-order", - "get-order", // no operationId → intent name from the route shape + "get-order", // no operationId -> intent name from the route shape "delete-order", ]); }); diff --git a/packages/codegen/src/sources/openapi.ts b/packages/codegen/src/sources/openapi.ts index c8f99e5..48c4eb6 100644 --- a/packages/codegen/src/sources/openapi.ts +++ b/packages/codegen/src/sources/openapi.ts @@ -7,10 +7,10 @@ * changing any application code. * * What we read from each operation: - * - name ← operationId, slugified (falls back to method + path) - * - description ← summary, else the first line of description, else a template - * - inputSchema ← path + query parameters merged with the JSON request body - * - outputSchema ← the first 2xx response's JSON schema, when present + * - name <- operationId, slugified (falls back to method + path) + * - description <- summary, else the first line of description, else a template + * - inputSchema <- path + query parameters merged with the JSON request body + * - outputSchema <- the first 2xx response's JSON schema, when present * * Header and cookie parameters are skipped on purpose: agents should not be * setting those by hand, and auth headers are the app's job, not the tool's. @@ -226,7 +226,7 @@ function buildInputSchema( for (const key of body.schema.required ?? []) required.add(key); } } else { - // A non-object body (array, raw string, …) goes under a "body" field. + // A non-object body (array, raw string, ...) goes under a "body" field. properties.body = body.schema; if (body.required) required.add("body"); } diff --git a/packages/codegen/src/sources/schema.ts b/packages/codegen/src/sources/schema.ts index 9b29a9c..ae51627 100644 --- a/packages/codegen/src/sources/schema.ts +++ b/packages/codegen/src/sources/schema.ts @@ -69,7 +69,7 @@ function isStandardSchema(value: unknown): value is StandardSchemaV1 { /** * TypeBox v1 schemas are JSON Schema documents, not wrapper objects: the value * has `type` and `properties` and no `~standard`. This is the only check that - * does not lie about what TypeBox is — there is no vendor marker to read. + * does not lie about what TypeBox is - there is no vendor marker to read. */ function isTypeBoxSchema(value: unknown): value is JsonSchema { if (typeof value !== "object" || value === null || Array.isArray(value)) return false; @@ -159,7 +159,7 @@ function toJsonSchema( anchorDir: string, ): JsonSchema { // TypeBox schemas are JSON Schema already: `Type.Object({...})` returns the - // draft-2020-12 shape with no wrapper. Detected by shape, not a marker — + // draft-2020-12 shape with no wrapper. Detected by shape, not a marker - // TypeBox v1 has no `~standard` and no `_def`; the object is the contract. if (isTypeBoxSchema(value)) { return value; @@ -230,37 +230,3 @@ function toJsonSchema( ); } } - -/** - * `generate --suggest` support: filter a module's exports to Standard Schemas - * and convert each to JSON Schema text for the proposal prompt. Lives next to - * the source so the conversion rules exist exactly once. Exports that are not - * schemas are skipped silently (a module may export anything); exports that - * look like schemas but fail conversion are reported, never silent. - */ -export function schemaExportsToJson( - moduleExports: Record, - anchorDir: string, -): { - schemas: { name: string; schemaText: string }[]; - skipped: { name: string; reason: string }[]; -} { - const schemas: { name: string; schemaText: string }[] = []; - const skipped: { name: string; reason: string }[] = []; - for (const [name, value] of Object.entries(moduleExports)) { - // Standard Schema (zod/valibot/arktype) carries a `~standard` marker; - // TypeBox is a bare JSON-Schema object. Both are valid — this loader was - // the one place that only accepted the marker, which is why TypeBox - // modules silently yielded nothing. - if (!isStandardSchema(value) && !isTypeBoxSchema(value)) continue; - try { - schemas.push({ - name, - schemaText: JSON.stringify(toJsonSchema(value, name, "schema", anchorDir)), - }); - } catch (error) { - skipped.push({ name, reason: error instanceof Error ? error.message : String(error) }); - } - } - return { schemas, skipped }; -} diff --git a/packages/codegen/src/types.ts b/packages/codegen/src/types.ts index 4ebdd2e..d24b749 100644 --- a/packages/codegen/src/types.ts +++ b/packages/codegen/src/types.ts @@ -3,7 +3,7 @@ * * Every stage of the pipeline speaks in these types: * - * Source → CandidateTool → (safety review) → ReviewedTool → Output → GeneratedFile + * Source -> CandidateTool -> (safety review) -> ReviewedTool -> Output -> GeneratedFile * * A source only has to produce CandidateTools. An output only has to turn * ReviewedTools into files. Everything in between lives here so the stages @@ -93,6 +93,25 @@ export interface CandidateTool { * the route carries the auth/admin/destructive signal. */ endpointRef?: string; + /** + * For a grouped handshake tool: the two calls and how the first response + * feeds the second's inputs. Absent for ordinary single-endpoint tools. + */ + compose?: { + first: { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + }; + second: { + httpMethod: NonNullable; + pathTemplate: string; + paramLocations: { path: string[]; query: string[]; body: string[] }; + serverUrl?: string; + }; + threaded: Record; + }; /** * Present when the tool annotates a literal component instead of * generating a .webmcp.ts file. Set by the schema source from the entry's @@ -256,56 +275,4 @@ export interface CodegenConfig { sources: Source[]; outputs: Output[]; safety?: SafetyOptions; - /** - * The opt-in LLM layer. Absent means off: the run is then exactly the - * deterministic one. The layer only ever proposes (report lines); it never - * writes files, never classifies risk, and never changes exit codes. - */ - llm?: LlmOptions; -} - -/** The four things the LLM layer may propose on. */ -export type LlmTask = "describe" | "relationship" | "semantic-review" | "suggest"; - -/** - * A model backend. Bring your own to use any vendor, or configure a key and - * use the built-in OpenAI-compatible one. One method, because the layer asks - * one kind of question. - */ -export interface LlmProvider { - name: string; - /** - * Ask one question. `system` is the per-task prompt (the built-in default, - * or the developer's override from config); providers that ignore it fall - * back to whatever the task implies. - */ - complete(task: LlmTask, prompt: string, system?: string): Promise; -} - -export interface LlmOptions { - /** A custom provider. Wins over apiKey when both are set. */ - provider?: LlmProvider; - /** API key for the built-in provider. Falls back to env WEBMCP_LLM_API_KEY, then OPENAI_API_KEY. */ - apiKey?: string; - /** OpenAI-compatible base URL. Default: https://api.openai.com/v1 */ - baseUrl?: string; - /** Model name for the built-in provider. Default: gpt-4o-mini */ - model?: string; - /** Override the shipped prompt per task, e.g. to match your domain's voice. */ - prompts?: Partial>; -} - -/** - * One proposal from the LLM layer. Rendered as `◦` lines, visually apart from - * audit findings, because a suggestion is not a fact: the developer disposes. - * Nothing here is ever applied to a file in this version; the acceptance - * surface (dashboard accept/reject) is a deliberate follow-up. - */ -export interface LlmSuggestion { - task: LlmTask; - /** The tool (and field, when relevant) this proposal is about. */ - tool?: string; - field?: string; - /** The one-line proposal text for the report. */ - message: string; } diff --git a/packages/codegen/src/verify.test.ts b/packages/codegen/src/verify.test.ts new file mode 100644 index 0000000..c5394f7 --- /dev/null +++ b/packages/codegen/src/verify.test.ts @@ -0,0 +1,198 @@ +import { describe, expect, it } from "vitest"; +import type { ReviewedTool } from "./types.js"; +import { countJourneyTools, verifyJourneyFiles, verifyTools } from "./verify.js"; + +function reviewedTool(overrides: Partial = {}): ReviewedTool { + return { + id: "GET /orders/{id}", + name: "get-order-status", + source: { kind: "openapi", ref: "GET /orders/{id}" }, + inputSchema: { + type: "object", + properties: { orderId: { type: "string", description: "The order ID" } }, + required: ["orderId"], + }, + description: "Returns the current status and tracking info for an order by ID.", + descriptionSource: "openapi-summary", + sideEffect: "read", + enabledByDefault: true, + withheld: false, + riskTier: "safe-read", + hints: { readOnlyHint: true, untrustedContentHint: false }, + ...overrides, + } as ReviewedTool; +} + +describe("verify description budgets", () => { + it("warns (not errors) when a tool description is over the 500-character budget", () => { + const checks = verifyTools([reviewedTool({ description: `List. ${"word ".repeat(120)}` })]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("warning"); + expect(budgets?.findings[0]).toContain("get-order-status"); + expect(budgets?.findings[0]).toContain("500"); + }); + + it("warns on a parameter description over the 150-character budget, with its field", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { notes: { type: "string", description: `Notes. ${"word ".repeat(40)}` } }, + }, + }), + ]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("warning"); + expect(budgets?.findings[0]).toContain("get-order-status -> notes"); + expect(budgets?.findings[0]).toContain("150"); + }); + + it("walks nested fields for the parameter budget", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { + meta: { + type: "object", + properties: { remark: { type: "string", description: `R. ${"word ".repeat(40)}` } }, + }, + }, + }, + }), + ]); + const budgets = checks.find((check) => check.area === "Budgets"); + expect(budgets?.level).toBe("warning"); + expect(budgets?.findings[0]).toContain("remark"); + }); + + it("is silent when everything is within budget", () => { + const checks = verifyTools([reviewedTool()]); + expect(checks.find((check) => check.area === "Budgets")).toBeUndefined(); + }); + + it("flags parameter names over 30 characters as a naming warning", () => { + const checks = verifyTools([ + reviewedTool({ + inputSchema: { + type: "object", + properties: { + the_unreasonably_long_parameter_name: { type: "string", description: "An id." }, + }, + }, + }), + ]); + const names = checks.find((check) => check.area === "Names"); + expect(names?.level).toBe("warning"); + expect(names?.findings[0]).toContain("the_unreasonably_long_parameter_name"); + }); +}); + +describe("verify journey files", () => { + const goodJourney = `import { createJourney } from "../journey.webmcp"; +export const documentTrip = createJourney({ + name: "document-trip", + goal: "Record a trip you've been on and open the editor to write its story", + steps: { + "search-places": { + description: "Search real places and store the pick.", + input: { type: "object", properties: { input: { type: "string" } }, required: ["input"] }, + provides: ["locationObject"], + }, + }, + submit: { + description: "Create the trip and open it in the editor.", + build: (draft) => draft, + run: executeCreateTrip, + }, +});`; + + it("passes a well-formed journey", () => { + const checks = verifyJourneyFiles([ + { path: "journeys/document-trip.webmcp.ts", contents: goodJourney }, + ]); + expect(checks).toHaveLength(1); + expect(checks[0]?.level).toBe("ok"); + }); + + it("errors on a journeys file with no createJourney call", () => { + const checks = verifyJourneyFiles([ + { path: "journeys/random.webmcp.ts", contents: "export const x = 1;" }, + ]); + expect(checks[0]?.level).toBe("error"); + expect(checks[0]?.findings[0]).toContain("no createJourney() call"); + }); + + it("errors on a journey with no submit gate", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/broken.webmcp.ts", + contents: `createJourney({ name: "x", goal: "y", steps: { "a": { description: "Do a.", input: {}, provides: [] } } });`, + }, + ]); + expect(checks.some((check) => check.findings[0]?.includes("no submit gate"))).toBe(true); + }); + + it("warns on over-budget descriptions", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/wordy.webmcp.ts", + contents: goodJourney.replace( + "Search real places and store the pick.", + `Search. ${"So very much detail about searching. ".repeat(15)}`, + ), + }, + ]); + const budget = checks.find( + (check) => check.level === "warning" && check.summary.includes("budget"), + ); + expect(budget?.findings[0]).toContain("500"); + }); + + it("counts the tools a journey registers: one per step plus the submit gate", () => { + // goodJourney has one step and one submit, so it registers two tools. + expect(countJourneyTools([{ path: "journeys/a.webmcp.ts", contents: goodJourney }])).toBe(2); + }); + + it("adds journey tools to the surface total and warns past the budget", () => { + const endpointTools = Array.from({ length: 24 }, (_, i) => + reviewedTool({ name: `list-thing-${i}`, withheld: false }), + ); + // 24 endpoints alone are under 25; a journey pushes the real surface past it. + const checks = verifyTools(endpointTools, { journeyToolCount: 4 }); + const surface = checks.find((check) => check.area === "Surface"); + expect(surface?.level).toBe("warning"); + expect(surface?.summary).toContain("28"); + expect(surface?.summary).toContain("4 journey"); + }); + + it("warns on more than five steps", () => { + const steps = Array.from( + { length: 6 }, + (_, i) => + `"step-${i}": { description: "Do thing ${i}.", input: { type: "object" }, provides: ["f${i}"] },`, + ).join("\n "); + const checks = verifyJourneyFiles([ + { + path: "journeys/epic.webmcp.ts", + contents: `createJourney({ name: "epic", goal: "g", steps: {\n ${steps}\n }, submit: { description: "Go.", build: (d) => d, run: x } });`, + }, + ]); + const warning = checks.find((check) => check.level === "warning"); + expect(warning?.findings[0]).toContain("6 steps"); + }); + + it("warns when a journey calls fetch directly", () => { + const checks = verifyJourneyFiles([ + { + path: "journeys/raw.webmcp.ts", + contents: goodJourney.replace( + 'provides: ["locationObject"],', + 'provides: ["locationObject"],\n run: async (input) => { const r = await fetch("/v1/places"); return { locationObject: r }; },', + ), + }, + ]); + const warning = checks.find((check) => check.level === "warning"); + expect(warning?.findings[0]).toContain("fetch/callApi directly"); + }); +}); diff --git a/packages/codegen/src/verify.ts b/packages/codegen/src/verify.ts index 5fcc331..ebdab72 100644 --- a/packages/codegen/src/verify.ts +++ b/packages/codegen/src/verify.ts @@ -3,8 +3,8 @@ * * Why this exists: a generated surface's quality was only ever discovered * after deploy, by an external audit reading the page's registry. Verify runs - * the same rubric over the tools a run would register — name shape, what the - * description covers, field text, annotations, surface size — and prints a + * the same rubric over the tools a run would register - name shape, what the + * description covers, field text, annotations, surface size - and prints a * scorecard before anything ships. * * The checks are deterministic: same tools in, same verdict out, no key, no @@ -12,11 +12,15 @@ * findings so CI can gate on it. */ +import { FIELD_DESCRIPTION_MAX, TOOL_DESCRIPTION_MAX } from "./describe.js"; import type { AuditFinding, ReviewedTool } from "./types.js"; /** Chrome's published guidance for tool names. */ const NAME_MAX = 30; +/** Chrome's budget for parameter names (same as tool names). */ +const PARAM_NAME_MAX = 30; + /** Verbs a good tool name starts with. Method verbs plus the action words * the naming algorithm knows; the list lives here because verify's whole * job is judging names from the outside. */ @@ -128,6 +132,28 @@ function check(area: string, offenders: string[], okText: string): VerifyCheck { }; } +/** + * Walk every input field of a schema - nested objects and array items, two + * levels down, mirroring the describe layer's coverage - and run `visit` on + * each name and its text. + */ +function eachField( + schema: { + properties?: Record; + items?: unknown; + }, + depth: number, + visit: (name: string, description: string | undefined) => void, +): void { + if (depth > 2) return; + for (const [name, field] of Object.entries(schema.properties ?? {})) { + visit(name, typeof field.description === "string" ? field.description : undefined); + eachField(field as typeof schema, depth + 1, visit); + const items = (field as { items?: unknown }).items; + if (items && typeof items === "object") eachField(items as typeof schema, depth + 1, visit); + } +} + /** True when any field (including nested ones, two levels down) lacks text. * The root schema is the object itself, not a field, so it is not checked. */ function hasBareField( @@ -157,42 +183,68 @@ function hasBareField( * each check serves is named in its area, so the scorecard maps to the * design doc's table without translation. */ -export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { +export function verifyTools( + tools: ReviewedTool[], + options?: { journeyToolCount?: number }, +): VerifyCheck[] { const registered = tools.filter((tool) => !tool.withheld); const longNames = registered .filter((tool) => tool.name.length > NAME_MAX) .map( - (tool) => `${tool.name} (${tool.name.length} chars) — rename it in the dashboard or config.`, + (tool) => `${tool.name} (${tool.name.length} chars) - rename it in the dashboard or config.`, ); + const longParamNames: string[] = []; + const longParamDescriptions: string[] = []; + for (const tool of registered) { + eachField(tool.inputSchema, 0, (name, description) => { + if (name.length > PARAM_NAME_MAX) { + longParamNames.push( + `${tool.name} -> ${name} (${name.length} chars) - parameter names max ${PARAM_NAME_MAX}.`, + ); + } + if (description && description.length > FIELD_DESCRIPTION_MAX) { + longParamDescriptions.push( + `${tool.name} -> ${name} (${description.length} chars) - over the ${FIELD_DESCRIPTION_MAX}-character parameter budget; tighten it.`, + ); + } + }); + } const nonVerb = registered .filter((tool) => !KNOWN_VERBS.has(tool.name.split("-")[0] ?? "")) - .map((tool) => `${tool.name} — agents pick tools by their first word; lead with the action.`); + .map((tool) => `${tool.name} - agents pick tools by their first word; lead with the action.`); + + const longDescriptions = registered + .filter((tool) => tool.description && tool.description.length > TOOL_DESCRIPTION_MAX) + .map( + (tool) => + `${tool.name} (${tool.description.length} chars) - over the ${TOOL_DESCRIPTION_MAX}-character tool budget; tighten it.`, + ); const noDescription = registered .filter((tool) => !tool.description || tool.description.trim() === "") - .map((tool) => `${tool.name} — no description; the tool is invisible to agents.`); + .map((tool) => `${tool.name} - no description; the tool is invisible to agents.`); const templateDescription = registered .filter((tool) => tool.descriptionSource === "generated-template") - .map((tool) => `${tool.name} — description is a machine draft; write the real one.`); + .map((tool) => `${tool.name} - description is a machine draft; write the real one.`); const noReturnShape = registered .filter( (tool) => tool.outputSchema && tool.description && !RETURN_LANGUAGE.test(tool.description), ) - .map((tool) => `${tool.name} — the description never says what comes back.`); + .map((tool) => `${tool.name} - the description never says what comes back.`); const bareFields = registered .filter((tool) => hasBareField(tool.inputSchema, 0)) - .map((tool) => `${tool.name} — an input field has no description an agent can act on.`); + .map((tool) => `${tool.name} - an input field has no description an agent can act on.`); const readWithoutHint = registered .filter((tool) => tool.sideEffect === "read" && !tool.hints.readOnlyHint) - .map((tool) => `${tool.name} — a read without readOnlyHint looks unsafe to call.`); + .map((tool) => `${tool.name} - a read without readOnlyHint looks unsafe to call.`); const checks: VerifyCheck[] = [ check( "Names", - [...longNames, ...nonVerb], + [...longNames, ...longParamNames, ...nonVerb], `all ${registered.length} names within 30 characters, verb-first`, ), check( @@ -204,6 +256,20 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { check("Annotations", readWithoutHint, "reads declare readOnlyHint; content declares its trust"), ]; + // Chrome's character budgets are authoring guidance, not spec rules: the + // browser only rejects an empty description or a bad name. So an overrun is + // a warning that points at a shorter rewrite, never an error that blocks CI. + // (A long description is a quality smell, not a broken tool.) + const budgetOffenders = [...longDescriptions, ...longParamDescriptions]; + if (budgetOffenders.length > 0) { + checks.push({ + area: "Budgets", + summary: `${budgetOffenders.length} over budget`, + findings: budgetOffenders, + level: "warning", + }); + } + // Missing descriptions are the one error: the audit treats them as fatal, // and so do we. if (noDescription.length > 0) { @@ -215,13 +281,23 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { }); } - if (registered.length > 25) { + // Journey tools register at runtime, so they are counted by the caller and + // passed in: the surface an agent actually sees is endpoint tools plus + // journey steps plus their submit gates. Missing this would let a journey + // quietly grow the surface past the budget the docs promise is watched. + const journeyToolCount = options?.journeyToolCount ?? 0; + const surfaceTotal = registered.length + journeyToolCount; + if (surfaceTotal > 25) { + const breakdown = + journeyToolCount > 0 + ? `${registered.length} endpoint and ${journeyToolCount} journey` + : `${registered.length}`; checks.push({ area: "Surface", - summary: `${registered.length} registered`, + summary: `${surfaceTotal} registered (${breakdown})`, findings: [ - `${registered.length} tools register on this surface — agents choose measurably worse past a handful. ` + - "Withhold unreviewed tools, or narrow with safety.exclude.", + `${surfaceTotal} tools register on this surface (${breakdown}) - agents choose measurably worse past a handful. ` + + "Withhold unreviewed tools, split journeys, or narrow with safety.exclude.", ], level: "warning", }); @@ -230,6 +306,166 @@ export function verifyTools(tools: ReviewedTool[]): VerifyCheck[] { return checks; } +/** + * The top-level keys of an inline object literal's `steps: { ... }` block, + * found by brace matching rather than parsing (journey files are the user's + * TypeScript; a full parse is out of scope for a lint). Used to count steps. + */ +function journeyStepNames(contents: string): string[] { + const start = /steps\s*:\s*\{/.exec(contents); + if (!start) return []; + let depth = 0; + let bodyStart = -1; + let bodyEnd = -1; + for (let i = start.index + start[0].length - 1; i < contents.length; i++) { + const char = contents[i]; + if (char === "{") { + if (depth === 0) bodyStart = i + 1; + depth++; + } else if (char === "}") { + depth--; + if (depth === 0) { + bodyEnd = i; + break; + } + } + } + if (bodyStart === -1 || bodyEnd === -1) return []; + const body = contents.slice(bodyStart, bodyEnd); + // Keys at depth one: "search-places": { ... }. The sticky regex anchors at + // the first non-space character after the walk position, and the walk + // skips past each match, so a key is counted exactly once. + const names: string[] = []; + const keyPattern = /"([^"]+)"\s*:\s*\{/y; + let innerDepth = 0; + for (let i = 0; i < body.length; i++) { + const char = body[i]; + if (char === "{") { + innerDepth++; + continue; + } + if (char === "}") { + innerDepth--; + continue; + } + if (innerDepth !== 0) continue; + let cursor = i; + while (cursor < body.length && /\s/.test(body[cursor] ?? "")) cursor++; + keyPattern.lastIndex = cursor; + const keyMatch = keyPattern.exec(body); + if (keyMatch) { + names.push(keyMatch[1] ?? ""); + // The match consumed the step's opening brace - count it, since the + // walk skips past the whole match including that brace. + innerDepth++; + i = cursor + keyMatch[0].length - 1; + } + } + return names; +} + +/** A journey definition file from the tools directory's journeys/ folder. */ +export interface JourneyFileInput { + path: string; + contents: string; +} + +/** + * The journey checks, over the user's journey files (not the generated + * tools). Structural problems (no createJourney, no submit gate, missing + * run) and over-budget descriptions are errors; step count and bypassing + * the generated callers are warnings. Journey files are the user's agent's + * code, so findings name the fix, not just the smell. + */ +export function verifyJourneyFiles(files: JourneyFileInput[]): VerifyCheck[] { + const structural: string[] = []; + const budget: string[] = []; + const warnings: string[] = []; + + for (const { path, contents } of files) { + if (!/createJourney\s*\(/.test(contents)) { + structural.push( + `${path} - no createJourney() call; files in journeys/ must define a journey.`, + ); + continue; + } + if (!/submit\s*:/.test(contents) || !/run\s*:/.test(contents)) { + structural.push( + `${path} - no submit gate with a run; a journey without one is just loose tools.`, + ); + } + const steps = journeyStepNames(contents); + if (steps.length > 5) { + warnings.push( + `${path} - ${steps.length} steps; past five, agents lose the thread. Split it into two journeys.`, + ); + } + for (const match of contents.matchAll(/description\s*:\s*"((?:[^"\\]|\\.)*)"/g)) { + const text = match[1] ?? ""; + if (text.length > TOOL_DESCRIPTION_MAX) { + budget.push( + `${path} - a description runs ${text.length} characters (max ${TOOL_DESCRIPTION_MAX}); tighten it.`, + ); + } + } + const withoutImports = contents.replace(/^\s*import\s.*$/gm, ""); + if (/\b(?:fetch|callApi)\s*\(/.test(withoutImports)) { + warnings.push( + `${path} - calls fetch/callApi directly; use the generated raw callers (fetchX) or a tool's execute, so the contract lives in one place.`, + ); + } + } + + const checks: VerifyCheck[] = []; + if (structural.length > 0) { + checks.push({ + area: "Journeys", + summary: `${structural.length} structural problem${structural.length === 1 ? "" : "s"}`, + findings: structural, + level: "error", + }); + } + if (budget.length > 0) { + checks.push({ + area: "Journeys", + summary: `${budget.length} over budget`, + findings: budget, + level: "warning", + }); + } + if (warnings.length > 0) { + checks.push({ + area: "Journeys", + summary: `${warnings.length} warning${warnings.length === 1 ? "" : "s"}`, + findings: warnings, + level: "warning", + }); + } + if (checks.length === 0 && files.length > 0) { + checks.push({ + area: "Journeys", + summary: `${files.length} journey file${files.length === 1 ? "" : "s"}, gates and budgets in order`, + findings: [], + level: "ok", + }); + } + return checks; +} + +/** + * How many tools a set of journey files registers on the page: one per step + * plus the submit gate. Used by the surface check, because these tools are + * created at runtime and never pass through the pipeline's tool list. + */ +export function countJourneyTools(files: JourneyFileInput[]): number { + let count = 0; + for (const { contents } of files) { + if (!/createJourney\s*\(/.test(contents)) continue; + count += journeyStepNames(contents).length + 1; + } + return count; +} + /** The --url probe: is the page actually live for a visitor's browser? */ export async function verifyUrl(url: string): Promise { const findings: AuditFinding[] = []; diff --git a/packages/codegen/src/wire.ts b/packages/codegen/src/wire.ts index 00d56f3..100ebfc 100644 --- a/packages/codegen/src/wire.ts +++ b/packages/codegen/src/wire.ts @@ -3,7 +3,7 @@ * * Generated files do nothing until something calls registerAllTools() once at * startup. Rather than telling the developer to go do that, we do it for - * them — under strict rules, because this is the one place we edit *their* + * them - under strict rules, because this is the one place we edit *their* * files instead of ours: * * 1. Edits are additive only. We insert lines; we never change or remove @@ -67,16 +67,16 @@ export async function applyWiring(plan: WirePlan): Promise { } } -/* ── Next.js (app router) ──────────────────────────────────────────────── */ +/* -- Next.js (app router) ------------------------------------------------ */ /** * Next needs the registration to run on the client, so we generate a tiny * "use client" component next to the tools and mount it in the root layout: * - * import { WebMCPRegister } from "../webmcp/register"; ← added + * import { WebMCPRegister } from "../webmcp/register"; <- added * ... * - * ← added + * <- added * {children} */ async function planNextWiring(cwd: string, app: WebApp, outDir: string): Promise { @@ -92,7 +92,7 @@ async function planNextWiring(cwd: string, app: WebApp, outDir: string): Promise const registerPath = join(cwd, outDir, "register.tsx"); const layout = await readFile(layoutPath, "utf8"); if (layout.includes("WebMCPRegister")) { - // The layout mounts the component — but "wired" also means the file it + // The layout mounts the component - but "wired" also means the file it // points at exists. A deleted register.tsx (or a fresh clone where it // was never committed) must not leave the app broken. try { @@ -165,13 +165,13 @@ export function WebMCPRegister() { `; } -/* ── Vite + React (SPAs) ───────────────────────────────────────────────── */ +/* -- Vite + React (SPAs) ------------------------------------------------- */ /** * A Vite app boots in main.tsx, so wiring is two added lines there: * - * import { registerAllTools } from "./webmcp"; ← added - * void registerAllTools(); ← added + * import { registerAllTools } from "./webmcp"; <- added + * void registerAllTools(); <- added */ async function planViteWiring(cwd: string, app: WebApp, outDir: string): Promise { const entryCandidates = [ @@ -205,7 +205,7 @@ async function planViteWiring(cwd: string, app: WebApp, outDir: string): Promise }; } -/* ── Shared helpers ────────────────────────────────────────────────────── */ +/* -- Shared helpers ------------------------------------------------------ */ /** Insert a line after the file's last top-level import statement. */ function insertAfterLastImport(source: string, line: string): string | null { @@ -221,7 +221,7 @@ function insertAfterLastImport(source: string, line: string): string | null { /** * Turn a filesystem path into a JS import specifier: no extension, and an - * explicit "./" when the target is in the same directory or deeper — + * explicit "./" when the target is in the same directory or deeper - * `relative()` alone yields "webmcp/index", which JS would read as a * package name, not a file. */ diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9cbb517..db306e7 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -96,9 +96,15 @@ importers: lucide-react: specifier: ^1.34.0 version: 1.34.0(react@19.2.8) + mermaid: + specifier: ^12.0.0 + version: 12.0.0 next: specifier: ^15.3.0 version: 15.5.24(@types/node@22.20.1)(react-dom@19.2.8(react@19.2.8))(react@19.2.8) + next-themes: + specifier: ^0.4.6 + version: 0.4.6(react-dom@19.2.8(react@19.2.8))(react@19.2.8) react: specifier: ^19.0.0 version: 19.2.8 @@ -137,6 +143,9 @@ packages: resolution: {integrity: sha512-UrcABB+4bUrFABwbluTIBErXwvbsU/V7TZWfmbgJfbkwiBuziS9gxdODUyuiecfdGQ85jglMW6juS3+z5TsKLw==} engines: {node: '>=10'} + '@antfu/install-pkg@2.0.1': + resolution: {integrity: sha512-iCKVQcIC0e3oDxEfs3SHQGW+ovhBMZmS1TE+bTk50rVyMCBmCfClv7Qi3HQKlumYwvjb/iIMeWCW2i67q6kFfQ==} + '@biomejs/biome@2.5.10': resolution: {integrity: sha512-WRKXARA3kTuiV5sxqTpobJ/I0MVd4vk3pOL6wnp5az4LntFIhWTj1RWZq3DI9PCEN3lXcqy7p5aqUHzvq8AXyQ==} engines: {node: '>=14.21.3'} @@ -205,6 +214,9 @@ packages: commander: optional: true + '@braintree/sanitize-url@7.1.2': + resolution: {integrity: sha512-jigsZK+sMF/cuiB7sERuo9V7N9jx+dhmHHnQyDSVdpZwVutaBu7WvNYqMDLSgFgfB30n452TP3vjDAvFC973mA==} + '@changesets/apply-release-plan@8.0.0': resolution: {integrity: sha512-kUd2pbf1w5/AYmBMb0Tt+rkIPCjFJdT0SZMrkOjJT/WV/QbtmvkyB5jkV0oaNPheavprZk+SfUiozUti7TIL2w==} engines: {node: ^22.11 || ^24 || >=26} @@ -266,6 +278,21 @@ packages: resolution: {integrity: sha512-q/ThtP9gcnEP6xlv7LrY26C2mHqdGBti/MmOVngt/M/oIGYkssmQGxPK9WzBNt2juVcH/vml2WQ+ra8LXYOTaA==} engines: {node: ^22.11 || ^24 || >=26} + '@chevrotain/cst-dts-gen@11.1.2': + resolution: {integrity: sha512-XTsjvDVB5nDZBQB8o0o/0ozNelQtn2KrUVteIHSlPd2VAV2utEb6JzyCJaJ8tGxACR4RiBNWy5uYUHX2eji88Q==} + + '@chevrotain/gast@11.1.2': + resolution: {integrity: sha512-Z9zfXR5jNZb1Hlsd/p+4XWeUFugrHirq36bKzPWDSIacV+GPSVXdk+ahVWZTwjhNwofAWg/sZg58fyucKSQx5g==} + + '@chevrotain/regexp-to-ast@11.1.2': + resolution: {integrity: sha512-nMU3Uj8naWer7xpZTYJdxbAs6RIv/dxYzkYU8GSwgUtcAAlzjcPfX1w+RKRcYG8POlzMeayOQ/znfwxEGo5ulw==} + + '@chevrotain/types@11.1.2': + resolution: {integrity: sha512-U+HFai5+zmJCkK86QsaJtoITlboZHBqrVketcO2ROv865xfCMSFpELQoz1GkX5GzME8pTa+3kbKrZHQtI0gdbw==} + + '@chevrotain/utils@11.1.2': + resolution: {integrity: sha512-4mudFAQ6H+MqBTfqLmU7G1ZwRzCLfJEooL/fsF6rCX5eePMbGhoy5n4g+G4vlh2muDcsCTJtL+uKbOzWxs5LHA==} + '@clack/core@1.4.3': resolution: {integrity: sha512-/kr3UWNtdJfxZtPgDqUOmG2pvwlmcLGheex5yiZKdwbzZJxhV+HMNR9QNmyY5cGwTNV6LrR7Jtp+KjhUAP1qBQ==} engines: {node: '>= 20.12.0'} @@ -611,6 +638,12 @@ packages: '@formatjs/intl-localematcher@0.6.2': resolution: {integrity: sha512-XOMO2Hupl0wdd172Y06h6kLpBz6Dv+J4okPLl4LPtzbr8f66WbIoy4ev98EBuZ6ZK4h5ydTN6XneT4QVpD7cdA==} + '@iconify/types@2.0.0': + resolution: {integrity: sha512-+wluvCrRhXrhyOmRDJ3q8mux9JkKy5SJ/v8ol2tu4FVjyYvtEzkc/3pK15ET6RKg4b4w4BmTk1+gsCUhf21Ykg==} + + '@iconify/utils@3.1.7': + resolution: {integrity: sha512-JZHlwdID+dy+lTgbYC8NEC4zeugqeYsc6jewvzb4c58kHauJn+X7rNwQjxz5p2qSjqaEeQoLkCIQ9v/H4PK0/w==} + '@img/colour@1.1.0': resolution: {integrity: sha512-Td76q7j57o/tLVdgS746cYARfSyxk8iEfRxewL9h4OMzYhbW4TAcppl0mT4eyqXddh6L/jwoM75mo7ixa/pCeQ==} engines: {node: '>=18'} @@ -788,6 +821,10 @@ packages: '@mdx-js/mdx@3.1.1': resolution: {integrity: sha512-f6ZO2ifpwAQIpzGWaBQT2TXxPv6z3RBzQKpVftEWN78Vl/YweF1uwussDx8ECAXVtr3Rs89fKyG9YlzUs9DyGQ==} + '@mermaid-js/parser@2.0.0': + resolution: {integrity: sha512-K8BeapFUfrfxbRAUQAG5oBOCwo1+bNWzaVnPxTCutkfrTBn6T/j91FIhqxWJ32SUeQ9T3iy1zcmPZ5ROZEvDrg==} + engines: {node: '>=22.12.0'} + '@napi-rs/lzma-linux-x64-gnu@1.5.1': resolution: {integrity: sha512-oTXEIha4SsuXdTA4Iyskj0kpdx2yVXdhd75c2v3xGrHFfVMsbhTPZU/nMPL4sWKo4pBHm3aucLaqGlF696dTyQ==} engines: {node: ^22.20 || ^24.12 || >=25} @@ -1491,6 +1528,99 @@ packages: '@types/chai@5.2.3': resolution: {integrity: sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==} + '@types/d3-array@3.2.2': + resolution: {integrity: sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==} + + '@types/d3-axis@3.0.6': + resolution: {integrity: sha512-pYeijfZuBd87T0hGn0FO1vQ/cgLk6E1ALJjfkC0oJ8cbwkZl3TpgS8bVBLZN+2jjGgg38epgxb2zmoGtSfvgMw==} + + '@types/d3-brush@3.0.6': + resolution: {integrity: sha512-nH60IZNNxEcrh6L1ZSMNA28rj27ut/2ZmI3r96Zd+1jrZD++zD3LsMIjWlvg4AYrHn/Pqz4CF3veCxGjtbqt7A==} + + '@types/d3-chord@3.0.6': + resolution: {integrity: sha512-LFYWWd8nwfwEmTZG9PfQxd17HbNPksHBiJHaKuY1XeqscXacsS2tyoo6OdRsjf+NQYeB6XrNL3a25E3gH69lcg==} + + '@types/d3-color@3.1.3': + resolution: {integrity: sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==} + + '@types/d3-contour@3.0.6': + resolution: {integrity: sha512-BjzLgXGnCWjUSYGfH1cpdo41/hgdWETu4YxpezoztawmqsvCeep+8QGfiY6YbDvfgHz/DkjeIkkZVJavB4a3rg==} + + '@types/d3-delaunay@6.0.4': + resolution: {integrity: sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==} + + '@types/d3-dispatch@3.0.7': + resolution: {integrity: sha512-5o9OIAdKkhN1QItV2oqaE5KMIiXAvDWBDPrD85e58Qlz1c1kI/J0NcqbEG88CoTwJrYe7ntUCVfeUl2UJKbWgA==} + + '@types/d3-drag@3.0.7': + resolution: {integrity: sha512-HE3jVKlzU9AaMazNufooRJ5ZpWmLIoc90A37WU2JMmeq28w1FQqCZswHZ3xR+SuxYftzHq6WU6KJHvqxKzTxxQ==} + + '@types/d3-dsv@3.0.7': + resolution: {integrity: sha512-n6QBF9/+XASqcKK6waudgL0pf/S5XHPPI8APyMLLUHd8NqouBGLsU8MgtO7NINGtPBtk9Kko/W4ea0oAspwh9g==} + + '@types/d3-ease@3.0.2': + resolution: {integrity: sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==} + + '@types/d3-fetch@3.0.7': + resolution: {integrity: sha512-fTAfNmxSb9SOWNB9IoG5c8Hg6R+AzUHDRlsXsDZsNp6sxAEOP0tkP3gKkNSO/qmHPoBFTxNrjDprVHDQDvo5aA==} + + '@types/d3-force@3.0.10': + resolution: {integrity: sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==} + + '@types/d3-format@3.0.4': + resolution: {integrity: sha512-fALi2aI6shfg7vM5KiR1wNJnZ7r6UuggVqtDA+xiEdPZQwy/trcQaHnwShLuLdta2rTymCNpxYTiMZX/e09F4g==} + + '@types/d3-geo@3.1.1': + resolution: {integrity: sha512-65Emv9fQiQQqphLlRkuQ5ypPsOmWPhtBGCMv61JDPEPMvsx+gzhGf74yw1a78xFKPj6zw4AgQICJoQv0vK9M2w==} + + '@types/d3-hierarchy@3.1.7': + resolution: {integrity: sha512-tJFtNoYBtRtkNysX1Xq4sxtjK8YgoWUNpIiUee0/jHGRwqvzYxkq0hGVbbOGSz+JgFxxRu4K8nb3YpG3CMARtg==} + + '@types/d3-interpolate@3.0.4': + resolution: {integrity: sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==} + + '@types/d3-path@3.1.1': + resolution: {integrity: sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==} + + '@types/d3-polygon@3.0.2': + resolution: {integrity: sha512-ZuWOtMaHCkN9xoeEMr1ubW2nGWsp4nIql+OPQRstu4ypeZ+zk3YKqQT0CXVe/PYqrKpZAi+J9mTs05TKwjXSRA==} + + '@types/d3-quadtree@3.0.6': + resolution: {integrity: sha512-oUzyO1/Zm6rsxKRHA1vH0NEDG58HrT5icx/azi9MF1TWdtttWl0UIUsjEQBBh+SIkrpd21ZjEv7ptxWys1ncsg==} + + '@types/d3-random@3.0.4': + resolution: {integrity: sha512-UHYId5WTCx4L4YNel7NU00XUXXgvgpgZOvp10PuvsQENjMDXhh2RyFc0KBjO7B45ne4Ha1yVH7ii0vnzKkuzWA==} + + '@types/d3-scale-chromatic@3.1.0': + resolution: {integrity: sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==} + + '@types/d3-scale@4.0.9': + resolution: {integrity: sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==} + + '@types/d3-selection@3.0.11': + resolution: {integrity: sha512-bhAXu23DJWsrI45xafYpkQ4NtcKMwWnAC/vKrd2l+nxMFuvOT3XMYTIj2opv8vq8AO5Yh7Qac/nSeP/3zjTK0w==} + + '@types/d3-shape@3.2.0': + resolution: {integrity: sha512-kVd74ta9eof3eJOvbNd1vGKS/XERRyQbT26Og63hIsvDO84cjD5gEOhsXf26w3FSoNlPVz84DOFcKv/oou+fMw==} + + '@types/d3-time-format@4.0.3': + resolution: {integrity: sha512-5xg9rC+wWL8kdDj153qZcsJ0FWiFt0J5RB6LYUNZjwSnesfblqrI/bJ1wBdJ8OQfncgbJG5+2F+qfqnqyzYxyg==} + + '@types/d3-time@3.0.4': + resolution: {integrity: sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==} + + '@types/d3-timer@3.0.2': + resolution: {integrity: sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==} + + '@types/d3-transition@3.0.9': + resolution: {integrity: sha512-uZS5shfxzO3rGlu0cC3bjmMFKsXv+SmZZcgp0KD22ts4uGXp5EVYGzu/0YdwZeKmddhcAccYtREJKkPfXkZuCg==} + + '@types/d3-zoom@3.0.8': + resolution: {integrity: sha512-iqMC4/YlFCSlO8+2Ii1GGGliCAY4XdeG748w5vQUbevlbDu0zSjH/+jojorQVBK/se0j6DUFNPBGSqD3YWYnDw==} + + '@types/d3@7.4.3': + resolution: {integrity: sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww==} + '@types/debug@4.1.13': resolution: {integrity: sha512-KSVgmQmzMwPlmtljOomayoR89W4FynCAi3E8PPs7vmDVPe84hT+vGPKkJfThkmXs0x0jAaa9U8uW8bbfyS2fWw==} @@ -1503,6 +1633,9 @@ packages: '@types/estree@1.0.9': resolution: {integrity: sha512-GhdPgy1el4/ImP05X05Uw4cw2/M93BCUmnEvWZNStlCzEKME4Fkk+YpoA5OiHNQmoS7Cafb8Xa3Pya8m1Qrzeg==} + '@types/geojson@7946.0.16': + resolution: {integrity: sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==} + '@types/hast@3.0.5': resolution: {integrity: sha512-rp/ezSWaD1m44dPKICGhiskI13nVr7qTloFwDa/IYkhhf5nzwP+zIQcIJh3WIFSBOy/H1PzB40jPjMDksN4F+g==} @@ -1532,6 +1665,9 @@ packages: '@types/react@19.2.18': resolution: {integrity: sha512-AnzbBERsrLKtk2XSfTbYRLjQPdy116Sty4q+T+Bp3IC4l6jNBvreVPAHmpq9qhXQM7CXZPjLVmGMw9sy+hxQ3w==} + '@types/trusted-types@2.0.7': + resolution: {integrity: sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==} + '@types/unist@2.0.11': resolution: {integrity: sha512-CmBKiL6NNo/OqgmMn95Fk9Whlp2mtvIv+KNpQKN2F4SjvrEesubTRWGYSg+BnWZOnlCaSTU1sMpsBOzgbYhnsA==} @@ -1547,6 +1683,9 @@ packages: '@ungap/structured-clone@1.3.4': resolution: {integrity: sha512-JL+CF0GeLHyPWI0rXu7UnxgiuOm9UQWzadi0OYOJNhNO2q6EZElpwlgXkNkfU1PzANDHq3YcwKVZprdvS+BrbQ==} + '@upsetjs/venn.js@2.0.0': + resolution: {integrity: sha512-WbBhLrooyePuQ1VZxrJjtLvTc4NVfpOyKx0sKqioq9bX1C1m7Jgykkn8gLrtwumBioXIqam8DLxp88Adbue6Hw==} + '@vitest/expect@4.1.11': resolution: {integrity: sha512-VX2x5vNJXET47KAFzwERI+KRMtTTCSWTfSMKsW7JsUsXV4psq++e3DvZpuTDOpHcxytiDs6p2nhVb2tVDiiUYw==} @@ -1660,6 +1799,9 @@ packages: character-reference-invalid@2.0.1: resolution: {integrity: sha512-iBZ4F4wRbyORVsu0jPV7gXkOsGYjGHPmAyv+HiHG8gi5PtC9KI2j1+v8/tlibRvjoWX027ypmG/n0HtO5t7unw==} + chevrotain@11.1.2: + resolution: {integrity: sha512-opLQzEVriiH1uUQ4Kctsd49bRoFDXGGSC4GUqj7pGyxM3RehRhvTlZJc1FL/Flew2p5uwxa1tUDWKzI4wNM8pg==} + chokidar@4.0.3: resolution: {integrity: sha512-Qgzu8kfBvo+cA4962jnP1KkS6Dop5NS6g7R5LFYJr4b8Ub94PPQXUksCw9PvXoeXPRRddRNC5C1JQUR2SMGtnA==} engines: {node: '>= 14.16.0'} @@ -1695,6 +1837,14 @@ packages: resolution: {integrity: sha512-NOKm8xhkzAjzFx8B2v5OAHT+u5pRQc2UCa2Vq9jYL/31o2wi9mxBA7LIFs3sV5VSC49z6pEhfbMULvShKj26WA==} engines: {node: '>= 6'} + commander@7.2.0: + resolution: {integrity: sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==} + engines: {node: '>= 10'} + + commander@8.3.0: + resolution: {integrity: sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==} + engines: {node: '>= 12'} + compute-scroll-into-view@3.1.1: resolution: {integrity: sha512-VRhuHOLoKYOy4UbilLbUzbYg93XLjv2PncJC50EuTWPA3gaja1UjBsUP/D/9/juV3vQFr6XBEzn9KCAHdUvOHw==} @@ -1712,6 +1862,12 @@ packages: convert-source-map@2.0.0: resolution: {integrity: sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==} + cose-base@1.0.3: + resolution: {integrity: sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==} + + cose-base@2.2.0: + resolution: {integrity: sha512-AzlgcsCbUMymkADOJtQm3wO9S3ltPfYOFD5033keQn9NJzIbtnZj+UdBJe7DYml/8TdbtHJW3j58SOnKhWY/5g==} + cssesc@3.0.0: resolution: {integrity: sha512-/Tb/JcjK111nNScGob5MNtsntNM1aCNUDipB/TkwZFhyDrrE47SOx/18wF2bbjgc3ZzCSKW1T5nt5EbFoAz/Vg==} engines: {node: '>=4'} @@ -1720,9 +1876,168 @@ packages: csstype@3.2.3: resolution: {integrity: sha512-z1HGKcYy2xA8AGQfwrn0PAy+PB7X/GSj3UVJW9qKyn43xWa+gl5nXmU4qqLMRzWVLFC8KusUX8T/0kCiOYpAIQ==} + cytoscape-cose-bilkent@4.1.0: + resolution: {integrity: sha512-wgQlVIUJF13Quxiv5e1gstZ08rnZj2XaLHGoFMYXz7SkNfCDOOteKBE6SYRfA9WxxI/iBc3ajfDoc6hb/MRAHQ==} + peerDependencies: + cytoscape: ^3.2.0 + + cytoscape-fcose@2.2.0: + resolution: {integrity: sha512-ki1/VuRIHFCzxWNrsshHYPs6L7TvLu3DL+TyIGEsRcvVERmxokbf5Gdk7mFxZnTdiGtnA4cfSmjZJMviqSuZrQ==} + peerDependencies: + cytoscape: ^3.2.0 + + cytoscape@3.34.3: + resolution: {integrity: sha512-yfYGhRcGAntq6YBD583j4n0Eg3jIxvWmZtz/5uz9UYkeIStSlMxuUja+ec5j3iBD8nv1rwaOAYMW09tBdkSeaQ==} + engines: {node: '>=0.10'} + + d3-array@2.12.1: + resolution: {integrity: sha512-B0ErZK/66mHtEsR1TkPEEkwdy+WDesimkM5gpZr5Dsg54BiTA5RXtYW5qTLIAcekaS9xfZrzBLF/OAkB3Qn1YQ==} + + d3-array@3.2.4: + resolution: {integrity: sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==} + engines: {node: '>=12'} + + d3-axis@3.0.0: + resolution: {integrity: sha512-IH5tgjV4jE/GhHkRV0HiVYPDtvfjHQlQfJHs0usq7M30XcSBvOotpmH1IgkcXsO/5gEQZD43B//fc7SRT5S+xw==} + engines: {node: '>=12'} + + d3-brush@3.0.0: + resolution: {integrity: sha512-ALnjWlVYkXsVIGlOsuWH1+3udkYFI48Ljihfnh8FZPF2QS9o+PzGLBslO0PjzVoHLZ2KCVgAM8NVkXPJB2aNnQ==} + engines: {node: '>=12'} + + d3-chord@3.0.1: + resolution: {integrity: sha512-VE5S6TNa+j8msksl7HwjxMHDM2yNK3XCkusIlpX5kwauBfXuyLAtNg9jCp/iHH61tgI4sb6R/EIMWCqEIdjT/g==} + engines: {node: '>=12'} + + d3-color@3.1.0: + resolution: {integrity: sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==} + engines: {node: '>=12'} + + d3-contour@4.0.2: + resolution: {integrity: sha512-4EzFTRIikzs47RGmdxbeUvLWtGedDUNkTcmzoeyg4sP/dvCexO47AaQL7VKy/gul85TOxw+IBgA8US2xwbToNA==} + engines: {node: '>=12'} + + d3-delaunay@6.0.4: + resolution: {integrity: sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==} + engines: {node: '>=12'} + + d3-dispatch@3.0.1: + resolution: {integrity: sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==} + engines: {node: '>=12'} + + d3-drag@3.0.0: + resolution: {integrity: sha512-pWbUJLdETVA8lQNJecMxoXfH6x+mO2UQo8rSmZ+QqxcbyA3hfeprFgIT//HW2nlHChWeIIMwS2Fq+gEARkhTkg==} + engines: {node: '>=12'} + + d3-dsv@3.0.1: + resolution: {integrity: sha512-UG6OvdI5afDIFP9w4G0mNq50dSOsXHJaRE8arAS5o9ApWnIElp8GZw1Dun8vP8OyHOZ/QJUKUJwxiiCCnUwm+Q==} + engines: {node: '>=12'} + hasBin: true + + d3-ease@3.0.1: + resolution: {integrity: sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==} + engines: {node: '>=12'} + + d3-fetch@3.0.1: + resolution: {integrity: sha512-kpkQIM20n3oLVBKGg6oHrUchHM3xODkTzjMoj7aWQFq5QEM+R6E4WkzT5+tojDY7yjez8KgCBRoj4aEr99Fdqw==} + engines: {node: '>=12'} + + d3-force@3.0.0: + resolution: {integrity: sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==} + engines: {node: '>=12'} + + d3-format@3.1.2: + resolution: {integrity: sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==} + engines: {node: '>=12'} + + d3-geo@3.1.1: + resolution: {integrity: sha512-637ln3gXKXOwhalDzinUgY83KzNWZRKbYubaG+fGVuc/dxO64RRljtCTnf5ecMyE1RIdtqpkVcq0IbtU2S8j2Q==} + engines: {node: '>=12'} + + d3-hierarchy@3.1.2: + resolution: {integrity: sha512-FX/9frcub54beBdugHjDCdikxThEqjnR93Qt7PvQTOHxyiNCAlvMrHhclk3cD5VeAaq9fxmfRp+CnWw9rEMBuA==} + engines: {node: '>=12'} + + d3-interpolate@3.0.1: + resolution: {integrity: sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==} + engines: {node: '>=12'} + + d3-path@1.0.9: + resolution: {integrity: sha512-VLaYcn81dtHVTjEHd8B+pbe9yHWpXKZUC87PzoFmsFrJqgFwDe/qxfp5MlfsfM1V5E/iVt0MmEbWQ7FVIXh/bg==} + + d3-path@3.1.0: + resolution: {integrity: sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==} + engines: {node: '>=12'} + + d3-polygon@3.0.1: + resolution: {integrity: sha512-3vbA7vXYwfe1SYhED++fPUQlWSYTTGmFmQiany/gdbiWgU/iEyQzyymwL9SkJjFFuCS4902BSzewVGsHHmHtXg==} + engines: {node: '>=12'} + + d3-quadtree@3.0.1: + resolution: {integrity: sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==} + engines: {node: '>=12'} + + d3-random@3.0.1: + resolution: {integrity: sha512-FXMe9GfxTxqd5D6jFsQ+DJ8BJS4E/fT5mqqdjovykEB2oFbTMDVdg1MGFxfQW+FBOGoB++k8swBrgwSHT1cUXQ==} + engines: {node: '>=12'} + + d3-sankey@0.12.3: + resolution: {integrity: sha512-nQhsBRmM19Ax5xEIPLMY9ZmJ/cDvd1BG3UVvt5h3WRxKg5zGRbvnteTyWAbzeSvlh3tW7ZEmq4VwR5mB3tutmQ==} + + d3-scale-chromatic@3.1.0: + resolution: {integrity: sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==} + engines: {node: '>=12'} + + d3-scale@4.0.2: + resolution: {integrity: sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==} + engines: {node: '>=12'} + + d3-selection@3.0.0: + resolution: {integrity: sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==} + engines: {node: '>=12'} + + d3-shape@1.3.7: + resolution: {integrity: sha512-EUkvKjqPFUAZyOlhY5gzCxCeI0Aep04LwIRpsZ/mLFelJiUfnK56jo5JMDSE7yyP2kLSb6LtF+S5chMk7uqPqw==} + + d3-shape@3.2.0: + resolution: {integrity: sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==} + engines: {node: '>=12'} + + d3-time-format@4.1.0: + resolution: {integrity: sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==} + engines: {node: '>=12'} + + d3-time@3.1.0: + resolution: {integrity: sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==} + engines: {node: '>=12'} + + d3-timer@3.0.1: + resolution: {integrity: sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==} + engines: {node: '>=12'} + + d3-transition@3.0.1: + resolution: {integrity: sha512-ApKvfjsSR6tg06xrL434C0WydLr7JewBB3V+/39RMHsaXTOG0zmt/OAXeng5M5LBm0ojmxJrpomQVZ1aPvBL4w==} + engines: {node: '>=12'} + peerDependencies: + d3-selection: 2 - 3 + + d3-zoom@3.0.0: + resolution: {integrity: sha512-b8AmV3kfQaqWAuacbPuNbL6vahnOJflOhexLzMMNLga62+/nh0JzvJ0aO/5a5MVgUFGS7Hu1P9P03o3fJkDCyw==} + engines: {node: '>=12'} + + d3@7.9.0: + resolution: {integrity: sha512-e1U46jVP+w7Iut8Jt8ri1YsPOvFpg46k+K8TpCb0P+zjCkjkPnV7WzfDJzMHy1LnA+wj5pLT1wjO901gLXeEhA==} + engines: {node: '>=12'} + + dagre-d3-es@7.0.14: + resolution: {integrity: sha512-P4rFMVq9ESWqmOgK+dlXvOtLwYg0i7u0HBGJER0LZDJT2VHIPAMZ/riPxqJceWMStH5+E61QxFra9kIS3AqdMg==} + dateformat@4.6.3: resolution: {integrity: sha512-2P0p0pFGzHS5EMnhdxQi7aJN+iMheud0UhG4dlE1DLAlvL8JHjJJTX/CSm4JXwV0Ka5nGk3zC5mcb5bUQUxxMA==} + dayjs@1.11.23: + resolution: {integrity: sha512-QDTCU0M0MxR3hQfnlDJfwekQiaanm1ubOD231u73WBckQ/fsamwRLiE2GBz6D3a/xF1NgfiDLJjXBa1hYOYTtQ==} + debug@4.4.3: resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==} engines: {node: '>=6.0'} @@ -1739,6 +2054,9 @@ packages: resolution: {integrity: sha512-cZTYKFWspt9jZsMscWo8sc/5lbPC9Q0N5nBLgb+Yd915iL3udB1uFgS3B8YCx66UVHq018DAVFoee7x+gxggeA==} engines: {node: '>=0.10.0'} + delaunator@5.1.0: + resolution: {integrity: sha512-AGrQ4QSgssa1NGmWmLPqN5NY2KajF5MqxetNEO+o0n3ZwZZeTmt7bBnvzHWrmkZFxGgr4HdyFgelzgi06otLuQ==} + dequal@2.0.3: resolution: {integrity: sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==} engines: {node: '>=6'} @@ -1753,6 +2071,12 @@ packages: devlop@1.1.0: resolution: {integrity: sha512-RWmIqhcFf1lRYBvNmr7qTNuyCt/7/ns2jbpp1+PalgE/rDQcBT0fioSMUpJ93irlUhC5hrg4cYqe6U+0ImW0rA==} + dompurify@3.4.15: + resolution: {integrity: sha512-EUBjM+B+lkDE41iE82DDSCfkoPGfXx8IxFxPMjNzm/Uk4xDet77rTN9wqlxlVg71kK7XGuUMv6wUxJUwwv+Xyw==} + + elkjs@0.9.3: + resolution: {integrity: sha512-f/ZeWvW/BCXbhGEf1Ujp29EASo/lk1FDnETgNKwJrsVvGZhUWCZyg3xLJjAsxfOmt8KjswHmI5EwCQcPMpOYhQ==} + emoji-regex@8.0.0: resolution: {integrity: sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A==} @@ -1770,6 +2094,9 @@ packages: es-module-lexer@2.3.2: resolution: {integrity: sha512-poHGpORABojJJucnV9KbOavETW8lBVnphkW77ER5/BQ5Fz7oXSoCNek7IH3vR5nRjdsEz926ibFYX8KtLQmdyw==} + es-toolkit@1.52.0: + resolution: {integrity: sha512-XTNEJQh1tY1ZJVcf6ayP/2n4ZPyaHlW2FWs7xvw5ddPuhUVjLD3olQVQS7kf58JbAB48iL0uL/jerTrjtV3lDA==} + esast-util-from-estree@2.0.0: resolution: {integrity: sha512-4CyanoAudUSBAn5K13H4JhsMH6L9ZP7XbLVe/dKybkxMO7eDyLsT8UHl9TRNrU2Gr9nz+FovfSIjuXWJ81uVwQ==} @@ -1939,6 +2266,9 @@ packages: graceful-fs@4.2.11: resolution: {integrity: sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ==} + hachure-fill@0.5.2: + resolution: {integrity: sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==} + happy-dom@20.11.6: resolution: {integrity: sha512-Hldbg8AdAa5a5oDcZpjqnGitp7JB0hqWmfv/8qr+kft4vzSD8BHsbdRfzYvL/0QcbKcURC/yyoygbeDQarPvYg==} engines: {node: '>=20.0.0'} @@ -1976,6 +2306,10 @@ packages: resolution: {integrity: sha512-zPGsiS+dWoTZtZ4AtpA9Y+BdSFSNWvnouNlWNoUFyAM6xHOHmdCvqO3k8AIbdamCOv4gUFUVNPf6rJFfc4UiJw==} hasBin: true + iconv-lite@0.6.3: + resolution: {integrity: sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==} + engines: {node: '>=0.10.0'} + image-size@2.0.2: resolution: {integrity: sha512-IRqXKlaXwgSMAMtpNzZa1ZAe8m+Sa1770Dhk8VkSsP9LS+iHD62Zd8FQKs8fbPiagBE7BzoFX23cxFnwshpV6w==} engines: {node: '>=16.x'} @@ -1987,6 +2321,13 @@ packages: inline-style-parser@0.2.7: resolution: {integrity: sha512-Nb2ctOyNR8DqQoR0OwRG95uNWIC0C1lCgf5Naz5H6Ji72KZ8OcFZLz2P5sNgwlyoJ8Yif11oMuYs5pBQa86csA==} + internmap@1.0.1: + resolution: {integrity: sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw==} + + internmap@2.0.3: + resolution: {integrity: sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==} + engines: {node: '>=12'} + is-accessor-descriptor@1.0.2: resolution: {integrity: sha512-AIbwAcazqP3R65dGvqk1V+a+vE5Fg1yu/ZKMOiBWSUIXXiwQkYmXQcVa2O0nh0tSDKDFKxG2mY7dB1Sr4hEP1g==} engines: {node: '>= 0.4'} @@ -2044,6 +2385,13 @@ packages: jsonc-parser@3.3.1: resolution: {integrity: sha512-HUgH65KyejrUFPvHFPbqOY0rsFip3Bo5wb4ngvdi1EpCYWUQDC5V+Y7mZws+DLkr4M//zQJoanu1SP+87Dv1oQ==} + katex@0.16.47: + resolution: {integrity: sha512-Eeo8Ys1doU1z+x8AZsPpQu+p/QcZBI5PeOo7QGQdy2x2m0MU/hYagBbGOmXwr5KVbEfVuWv9LpnQWeehogurjg==} + hasBin: true + + khroma@2.1.0: + resolution: {integrity: sha512-Ls993zuzfayK269Svk9hzpeGUKob/sIgZzyHYdjQoAdQetRKpOLj+k/QQQ/6Qi0Yz65mlROrfd+Ev+1+7dz9Kw==} + kind-of@3.2.2: resolution: {integrity: sha512-NOW9QQXMoZGg/oqnVNoNTTIFEIid1627WCffUBJEdMxYApq7mNE7CpzucIPc+ZQg25Phej7IJSmX3hO+oblOtQ==} engines: {node: '>=0.10.0'} @@ -2051,6 +2399,12 @@ packages: launch-editor@2.14.1: resolution: {integrity: sha512-QWBrQsMpH7gPr965dsKD/3cKWiNoTjpATQf++Xq63N6sKRGMwlVXz41O1IZTMfZQgBctD/K5Zt06+/I6pP6+HA==} + layout-base@1.0.2: + resolution: {integrity: sha512-8h2oVEZNktL4BH2JCOI90iD1yXwL6iNW7KcCKT2QZgQJR2vbqDsldCTPRU9NifTCqHZci57XvQQ15YTu+sTYPg==} + + layout-base@2.0.1: + resolution: {integrity: sha512-dp3s92+uNI1hWIpPGH3jK2kxE2lMjdXdr+DH8ynZHpd6PUlH6x6cbuXnoMmiNumznqaNO31xu9e79F0uuZ0JFg==} + lightningcss-android-arm64@1.32.0: resolution: {integrity: sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==} engines: {node: '>= 12.0.0'} @@ -2132,6 +2486,12 @@ packages: resolution: {integrity: sha512-IXO6OCs9yg8tMKzfPZ1YmheJbZCiEsnBdcB03l0OcfK9prKnJb96siuHCr5Fl37/yo9DnKU+TLpxzTUspw9shg==} engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + lodash-es@4.17.23: + resolution: {integrity: sha512-kVI48u3PZr38HdYz98UmfPnXl2DXrpdctLrFLCd3kOx1xUkOmpFPx7gCWWM5MPkL/fD8zb+Ph0QzjGFs4+hHWg==} + + lodash-es@4.18.1: + resolution: {integrity: sha512-J8xewKD/Gk22OZbhpOVSwcs60zhd95ESDwezOFuA3/099925PdHJ7OFHNTGtajL3AlZkykD32HykiMo+BIBI8A==} + lodash.merge@4.6.2: resolution: {integrity: sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ==} @@ -2157,6 +2517,11 @@ packages: markdown-table@3.0.4: resolution: {integrity: sha512-wiYz4+JrLyb/DqW2hkFJxP7Vd7JuTDm77fvbM8VfEQdmSMqcImWeeRbHwZjBjIFki/VaMK2BhFi7oUUZeM5bqw==} + marked@16.4.2: + resolution: {integrity: sha512-TI3V8YYWvkVf3KJe1dRkpnjs68JUPyEa5vjKrp1XEEJUAOaQc+Qj+L1qWbPd0SJuAdQkFU0h73sXXqwDYxsiDA==} + engines: {node: '>= 20'} + hasBin: true + mdast-util-find-and-replace@3.0.2: resolution: {integrity: sha512-Tmd1Vg/m3Xz43afeNxDIhWRtFZgM2VLyaf4vSTYwudTyeuTneoL3qtWMA5jeLyz/O1vDJmmV4QuScFCA2tBPwg==} @@ -2205,6 +2570,10 @@ packages: mdast-util-to-string@4.0.0: resolution: {integrity: sha512-0H44vDimn51F0YwvxSJSm0eCDOJTRlmN0R1yBh4HLj9wiV1Dn0QoXGbvFAWj2hSItVTlCmBF1hqKlIyUBVFLPg==} + mermaid@12.0.0: + resolution: {integrity: sha512-/wQXC9iBxoGV8p3erbvaXs9h77VyLDBH6GdayVjj3hEcSQhFU4N1WUhUppotCEqlIxI2pRMwjwBSwTB1MfZBgQ==} + engines: {node: '>=22.12.0'} + micromark-core-commonmark@2.0.3: resolution: {integrity: sha512-RDBrHEMSxVFLg6xvnXmb1Ayr2WzLAWjeSATAoxwKYJV94TeNavgoIdA0a9ytzDSVzBy2YKFK+emCPOEibLeCrg==} @@ -2389,6 +2758,9 @@ packages: parse-entities@4.0.2: resolution: {integrity: sha512-GG2AQYWoLgL877gQIKeRPGO1xF9+eG1ujIb5soS5gPvLQ1y2o8FL90w2QWNdf9I361Mpp7726c+lj3U0qK1uGw==} + path-data-parser@0.1.0: + resolution: {integrity: sha512-NOnmBpt5Y2RWbuv0LMzsayp3lVylAHLPUTut412ZA3l+C4uw4ZVkQbjShYCQ8TCpUMdPapr4YjUqLYD6v68j+w==} + path-to-regexp@8.4.2: resolution: {integrity: sha512-qRcuIdP69NPm4qbACK+aDogI5CBDMi1jKe0ry5rSQJz8JVLsC7jV8XpiJjGRLLol3N+R5ihGYcrPLTno6pAdBA==} @@ -2427,6 +2799,12 @@ packages: resolution: {integrity: sha512-Nc3IT5yHzflTfbjgqWcCPpo7DaKy4FnpB0l/zCAW0Tc7jxAiuqSxHasntB3D7887LSrA93kDJ9IXovxJYxyLCA==} engines: {node: '>=4'} + points-on-curve@0.2.0: + resolution: {integrity: sha512-0mYKnYYe9ZcqMCWhUjItv/oHjvgEsfKvnUTg8sAtnHr3GVy7rGkXCb6d5cSyqrWqL4k81b9CPg3urd+T7aop3A==} + + points-on-path@0.2.1: + resolution: {integrity: sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==} + postcss-load-config@6.0.1: resolution: {integrity: sha512-oPtTM4oerL+UXmx+93ytZVN82RrlY/wPUV8IeDxFrzIjXOLF1pN+EmKPLbubvKHT2HC20xXsCAH2Z+CKV6Oz/g==} engines: {node: '>= 18'} @@ -2573,15 +2951,27 @@ packages: resolution: {integrity: sha512-qYg9KP24dD5qka9J47d0aVky0N+b4fTU89LN9iDnjB5waksiC49rvMB0PrUJQGoTmH50XPiqOvAjDfaijGxYZw==} engines: {node: '>=8'} + robust-predicates@3.0.3: + resolution: {integrity: sha512-NS3levdsRIUOmiJ8FZWCP7LG3QpJyrs/TE0Zpf1yvZu8cAJJ6QMW92H1c7kWpdIHo8RvmLxN/o2JXTKHp74lUA==} + rollup@4.63.0: resolution: {integrity: sha512-T5vnZ2y4QqC3/4P+w2+JO+Q/OVdnPsv4XcSYJYMEn0R9/jjl5AgLwO9LAZMzP2lN71O6pypn91rB7lDstUkfrQ==} engines: {node: '>=18.0.0', npm: '>=8.0.0'} hasBin: true + roughjs@4.6.6: + resolution: {integrity: sha512-ZUz/69+SYpFN/g/lUlo2FXcIjRkSu3nDarreVdGGndHEBJ6cXPdKguS8JGxwj5HA5xIbVKSmLgr5b3AWxtRfvQ==} + + rw@1.3.3: + resolution: {integrity: sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==} + safe-stable-stringify@2.5.0: resolution: {integrity: sha512-b3rppTKm9T+PsVCBEOUR46GWI7fdOs00VKZ1+9c1EWDaDMvjQc6tUwuFyIprgGgTcWoVHSKrU8H31ZHA2e0RHA==} engines: {node: '>=10'} + safer-buffer@2.1.2: + resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==} + scheduler@0.27.0: resolution: {integrity: sha512-eNv+WrVbKu1f3vbYJT/xtiF5syA5HPIMtf9IgY/nKg0sWqzAUEvqY/xm7OcZc/qafLx/iO9FgOmeSAp4v5ti/Q==} @@ -2676,6 +3066,9 @@ packages: babel-plugin-macros: optional: true + stylis@4.4.0: + resolution: {integrity: sha512-5Z9ZpRzfuH6l/UAvCPAPUo3665Nk2wLaZU3x+TLHKVzIz33+sbJqbtrYoC3KD4/uVOr2Zp+L0LySezP9OHV9yA==} + sucrase@3.35.1: resolution: {integrity: sha512-DhuTmvZWux4H1UOnWMB3sk0sbaCVOoQZjv8u1rDoTV0HTdGem9hkAZtl4JZy8P2z4Bg0nT+YMeOFyVr4zcG5Tw==} engines: {node: '>=16 || 14 >=14.17'} @@ -2734,6 +3127,10 @@ packages: trough@2.2.0: resolution: {integrity: sha512-tmMpK00BjZiUyVyvrBK7knerNgmgvcV/KLVyuma/SC+TQN167GrMRciANTz09+k3zW8L8t60jWO1GpfkZdjTaw==} + ts-dedent@2.3.0: + resolution: {integrity: sha512-JfJeIHke7y2egdGGgRAvpCwYFUsHlM2gPcrVOxFkznt/4uzQ7HFmvE63iFHVLBJNDuyDOQgijDK/tXH/f6Msjg==} + engines: {node: '>=6.10'} + ts-interface-checker@0.1.13: resolution: {integrity: sha512-Y/arvbn+rrz3JCKl9C4kVNfTfSm2/mEp5FSz5EsZSANGPSlQrpRI5M4PKF+mJnE52jOO90PnPSc3Ur3bTQw0gA==} @@ -2824,6 +3221,10 @@ packages: util-deprecate@1.0.2: resolution: {integrity: sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==} + uuid@14.0.2: + resolution: {integrity: sha512-xZe/16rV4aa+HGSOCiY2YeLT1OybRLrrkL/Rqaq7p7GMVXjFh+6wN4oMYgjFmnSnhY8t6Xpdl2l9qmnHYuMHwQ==} + hasBin: true + vfile-message@4.0.3: resolution: {integrity: sha512-QTHzsGd1EhbZs4AsQ20JX1rC3cOlt/IWJruk893DfLRr57lcnOeMaWG4K0JrRta4mIJZKth2Au3mM3u03/JWKw==} @@ -2958,6 +3359,11 @@ snapshots: '@alloc/quick-lru@5.2.0': {} + '@antfu/install-pkg@2.0.1': + dependencies: + package-manager-detector: 1.8.0 + tinyexec: 1.3.0 + '@biomejs/biome@2.5.10': optionalDependencies: '@biomejs/cli-darwin-arm64': 2.5.10 @@ -2998,6 +3404,8 @@ snapshots: cac: 7.0.0 commander: 14.0.3 + '@braintree/sanitize-url@7.1.2': {} + '@changesets/apply-release-plan@8.0.0': dependencies: '@changesets/config': 4.0.0 @@ -3102,6 +3510,23 @@ snapshots: '@changesets/types': 7.0.0 human-id: 4.2.1 + '@chevrotain/cst-dts-gen@11.1.2': + dependencies: + '@chevrotain/gast': 11.1.2 + '@chevrotain/types': 11.1.2 + lodash-es: 4.17.23 + + '@chevrotain/gast@11.1.2': + dependencies: + '@chevrotain/types': 11.1.2 + lodash-es: 4.17.23 + + '@chevrotain/regexp-to-ast@11.1.2': {} + + '@chevrotain/types@11.1.2': {} + + '@chevrotain/utils@11.1.2': {} + '@clack/core@1.4.3': dependencies: fast-wrap-ansi: 0.2.2 @@ -3299,6 +3724,14 @@ snapshots: dependencies: tslib: 2.8.1 + '@iconify/types@2.0.0': {} + + '@iconify/utils@3.1.7': + dependencies: + '@antfu/install-pkg': 2.0.1 + '@iconify/types': 2.0.0 + import-meta-resolve: 4.2.0 + '@img/colour@1.1.0': optional: true @@ -3470,6 +3903,10 @@ snapshots: transitivePeerDependencies: - supports-color + '@mermaid-js/parser@2.0.0': + dependencies: + '@chevrotain/types': 11.1.2 + '@napi-rs/lzma-linux-x64-gnu@1.5.1': optional: true @@ -4076,6 +4513,123 @@ snapshots: '@types/deep-eql': 4.0.2 assertion-error: 2.0.1 + '@types/d3-array@3.2.2': {} + + '@types/d3-axis@3.0.6': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-brush@3.0.6': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-chord@3.0.6': {} + + '@types/d3-color@3.1.3': {} + + '@types/d3-contour@3.0.6': + dependencies: + '@types/d3-array': 3.2.2 + '@types/geojson': 7946.0.16 + + '@types/d3-delaunay@6.0.4': {} + + '@types/d3-dispatch@3.0.7': {} + + '@types/d3-drag@3.0.7': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-dsv@3.0.7': {} + + '@types/d3-ease@3.0.2': {} + + '@types/d3-fetch@3.0.7': + dependencies: + '@types/d3-dsv': 3.0.7 + + '@types/d3-force@3.0.10': {} + + '@types/d3-format@3.0.4': {} + + '@types/d3-geo@3.1.1': + dependencies: + '@types/geojson': 7946.0.16 + + '@types/d3-hierarchy@3.1.7': {} + + '@types/d3-interpolate@3.0.4': + dependencies: + '@types/d3-color': 3.1.3 + + '@types/d3-path@3.1.1': {} + + '@types/d3-polygon@3.0.2': {} + + '@types/d3-quadtree@3.0.6': {} + + '@types/d3-random@3.0.4': {} + + '@types/d3-scale-chromatic@3.1.0': {} + + '@types/d3-scale@4.0.9': + dependencies: + '@types/d3-time': 3.0.4 + + '@types/d3-selection@3.0.11': {} + + '@types/d3-shape@3.2.0': + dependencies: + '@types/d3-path': 3.1.1 + + '@types/d3-time-format@4.0.3': {} + + '@types/d3-time@3.0.4': {} + + '@types/d3-timer@3.0.2': {} + + '@types/d3-transition@3.0.9': + dependencies: + '@types/d3-selection': 3.0.11 + + '@types/d3-zoom@3.0.8': + dependencies: + '@types/d3-interpolate': 3.0.4 + '@types/d3-selection': 3.0.11 + + '@types/d3@7.4.3': + dependencies: + '@types/d3-array': 3.2.2 + '@types/d3-axis': 3.0.6 + '@types/d3-brush': 3.0.6 + '@types/d3-chord': 3.0.6 + '@types/d3-color': 3.1.3 + '@types/d3-contour': 3.0.6 + '@types/d3-delaunay': 6.0.4 + '@types/d3-dispatch': 3.0.7 + '@types/d3-drag': 3.0.7 + '@types/d3-dsv': 3.0.7 + '@types/d3-ease': 3.0.2 + '@types/d3-fetch': 3.0.7 + '@types/d3-force': 3.0.10 + '@types/d3-format': 3.0.4 + '@types/d3-geo': 3.1.1 + '@types/d3-hierarchy': 3.1.7 + '@types/d3-interpolate': 3.0.4 + '@types/d3-path': 3.1.1 + '@types/d3-polygon': 3.0.2 + '@types/d3-quadtree': 3.0.6 + '@types/d3-random': 3.0.4 + '@types/d3-scale': 4.0.9 + '@types/d3-scale-chromatic': 3.1.0 + '@types/d3-selection': 3.0.11 + '@types/d3-shape': 3.2.0 + '@types/d3-time': 3.0.4 + '@types/d3-time-format': 4.0.3 + '@types/d3-timer': 3.0.2 + '@types/d3-transition': 3.0.9 + '@types/d3-zoom': 3.0.8 + '@types/debug@4.1.13': dependencies: '@types/ms': 2.1.0 @@ -4088,6 +4642,8 @@ snapshots: '@types/estree@1.0.9': {} + '@types/geojson@7946.0.16': {} + '@types/hast@3.0.5': dependencies: '@types/unist': 3.0.3 @@ -4118,6 +4674,9 @@ snapshots: dependencies: csstype: 3.2.3 + '@types/trusted-types@2.0.7': + optional: true + '@types/unist@2.0.11': {} '@types/unist@3.0.3': {} @@ -4130,6 +4689,11 @@ snapshots: '@ungap/structured-clone@1.3.4': {} + '@upsetjs/venn.js@2.0.0': + optionalDependencies: + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + '@vitest/expect@4.1.11': dependencies: '@standard-schema/spec': 1.1.0 @@ -4227,6 +4791,15 @@ snapshots: character-reference-invalid@2.0.1: {} + chevrotain@11.1.2: + dependencies: + '@chevrotain/cst-dts-gen': 11.1.2 + '@chevrotain/gast': 11.1.2 + '@chevrotain/regexp-to-ast': 11.1.2 + '@chevrotain/types': 11.1.2 + '@chevrotain/utils': 11.1.2 + lodash-es: 4.17.23 + chokidar@4.0.3: dependencies: readdirp: 4.1.2 @@ -4256,6 +4829,10 @@ snapshots: commander@4.1.1: {} + commander@7.2.0: {} + + commander@8.3.0: {} + compute-scroll-into-view@3.1.1: {} confbox@0.1.8: {} @@ -4266,12 +4843,206 @@ snapshots: convert-source-map@2.0.0: {} + cose-base@1.0.3: + dependencies: + layout-base: 1.0.2 + + cose-base@2.2.0: + dependencies: + layout-base: 2.0.1 + cssesc@3.0.0: {} csstype@3.2.3: {} + cytoscape-cose-bilkent@4.1.0(cytoscape@3.34.3): + dependencies: + cose-base: 1.0.3 + cytoscape: 3.34.3 + + cytoscape-fcose@2.2.0(cytoscape@3.34.3): + dependencies: + cose-base: 2.2.0 + cytoscape: 3.34.3 + + cytoscape@3.34.3: {} + + d3-array@2.12.1: + dependencies: + internmap: 1.0.1 + + d3-array@3.2.4: + dependencies: + internmap: 2.0.3 + + d3-axis@3.0.0: {} + + d3-brush@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + + d3-chord@3.0.1: + dependencies: + d3-path: 3.1.0 + + d3-color@3.1.0: {} + + d3-contour@4.0.2: + dependencies: + d3-array: 3.2.4 + + d3-delaunay@6.0.4: + dependencies: + delaunator: 5.1.0 + + d3-dispatch@3.0.1: {} + + d3-drag@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-selection: 3.0.0 + + d3-dsv@3.0.1: + dependencies: + commander: 7.2.0 + iconv-lite: 0.6.3 + rw: 1.3.3 + + d3-ease@3.0.1: {} + + d3-fetch@3.0.1: + dependencies: + d3-dsv: 3.0.1 + + d3-force@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-quadtree: 3.0.1 + d3-timer: 3.0.1 + + d3-format@3.1.2: {} + + d3-geo@3.1.1: + dependencies: + d3-array: 3.2.4 + + d3-hierarchy@3.1.2: {} + + d3-interpolate@3.0.1: + dependencies: + d3-color: 3.1.0 + + d3-path@1.0.9: {} + + d3-path@3.1.0: {} + + d3-polygon@3.0.1: {} + + d3-quadtree@3.0.1: {} + + d3-random@3.0.1: {} + + d3-sankey@0.12.3: + dependencies: + d3-array: 2.12.1 + d3-shape: 1.3.7 + + d3-scale-chromatic@3.1.0: + dependencies: + d3-color: 3.1.0 + d3-interpolate: 3.0.1 + + d3-scale@4.0.2: + dependencies: + d3-array: 3.2.4 + d3-format: 3.1.2 + d3-interpolate: 3.0.1 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + + d3-selection@3.0.0: {} + + d3-shape@1.3.7: + dependencies: + d3-path: 1.0.9 + + d3-shape@3.2.0: + dependencies: + d3-path: 3.1.0 + + d3-time-format@4.1.0: + dependencies: + d3-time: 3.1.0 + + d3-time@3.1.0: + dependencies: + d3-array: 3.2.4 + + d3-timer@3.0.1: {} + + d3-transition@3.0.1(d3-selection@3.0.0): + dependencies: + d3-color: 3.1.0 + d3-dispatch: 3.0.1 + d3-ease: 3.0.1 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-timer: 3.0.1 + + d3-zoom@3.0.0: + dependencies: + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-interpolate: 3.0.1 + d3-selection: 3.0.0 + d3-transition: 3.0.1(d3-selection@3.0.0) + + d3@7.9.0: + dependencies: + d3-array: 3.2.4 + d3-axis: 3.0.0 + d3-brush: 3.0.0 + d3-chord: 3.0.1 + d3-color: 3.1.0 + d3-contour: 4.0.2 + d3-delaunay: 6.0.4 + d3-dispatch: 3.0.1 + d3-drag: 3.0.0 + d3-dsv: 3.0.1 + d3-ease: 3.0.1 + d3-fetch: 3.0.1 + d3-force: 3.0.0 + d3-format: 3.1.2 + d3-geo: 3.1.1 + d3-hierarchy: 3.1.2 + d3-interpolate: 3.0.1 + d3-path: 3.1.0 + d3-polygon: 3.0.1 + d3-quadtree: 3.0.1 + d3-random: 3.0.1 + d3-scale: 4.0.2 + d3-scale-chromatic: 3.1.0 + d3-selection: 3.0.0 + d3-shape: 3.2.0 + d3-time: 3.1.0 + d3-time-format: 4.1.0 + d3-timer: 3.0.1 + d3-transition: 3.0.1(d3-selection@3.0.0) + d3-zoom: 3.0.0 + + dagre-d3-es@7.0.14: + dependencies: + d3: 7.9.0 + lodash-es: 4.18.1 + dateformat@4.6.3: {} + dayjs@1.11.23: {} + debug@4.4.3: dependencies: ms: 2.1.3 @@ -4284,6 +5055,10 @@ snapshots: dependencies: is-descriptor: 1.0.4 + delaunator@5.1.0: + dependencies: + robust-predicates: 3.0.3 + dequal@2.0.3: {} detect-libc@2.1.2: {} @@ -4294,6 +5069,12 @@ snapshots: dependencies: dequal: 2.0.3 + dompurify@3.4.15: + optionalDependencies: + '@types/trusted-types': 2.0.7 + + elkjs@0.9.3: {} + emoji-regex@8.0.0: {} end-of-stream@1.4.5: @@ -4309,6 +5090,8 @@ snapshots: es-module-lexer@2.3.2: {} + es-toolkit@1.52.0: {} + esast-util-from-estree@2.0.0: dependencies: '@types/estree-jsx': 1.0.5 @@ -4550,6 +5333,8 @@ snapshots: graceful-fs@4.2.11: {} + hachure-fill@0.5.2: {} + happy-dom@20.11.6: dependencies: '@types/node': 26.3.0 @@ -4638,12 +5423,20 @@ snapshots: human-id@4.2.1: {} + iconv-lite@0.6.3: + dependencies: + safer-buffer: 2.1.2 + image-size@2.0.2: {} import-meta-resolve@4.2.0: {} inline-style-parser@0.2.7: {} + internmap@1.0.1: {} + + internmap@2.0.3: {} + is-accessor-descriptor@1.0.2: dependencies: hasown: 2.0.4 @@ -4690,6 +5483,12 @@ snapshots: jsonc-parser@3.3.1: {} + katex@0.16.47: + dependencies: + commander: 8.3.0 + + khroma@2.1.0: {} + kind-of@3.2.2: dependencies: is-buffer: 1.1.6 @@ -4699,6 +5498,10 @@ snapshots: picocolors: 1.1.1 shell-quote: 1.10.0 + layout-base@1.0.2: {} + + layout-base@2.0.1: {} + lightningcss-android-arm64@1.32.0: optional: true @@ -4754,6 +5557,10 @@ snapshots: load-tsconfig@0.2.5: {} + lodash-es@4.17.23: {} + + lodash-es@4.18.1: {} + lodash.merge@4.6.2: {} longest-streak@3.1.0: {} @@ -4772,6 +5579,8 @@ snapshots: markdown-table@3.0.4: {} + marked@16.4.2: {} + mdast-util-find-and-replace@3.0.2: dependencies: '@types/mdast': 4.0.4 @@ -4935,6 +5744,32 @@ snapshots: dependencies: '@types/mdast': 4.0.4 + mermaid@12.0.0: + dependencies: + '@braintree/sanitize-url': 7.1.2 + '@iconify/utils': 3.1.7 + '@mermaid-js/parser': 2.0.0 + '@types/d3': 7.4.3 + '@upsetjs/venn.js': 2.0.0 + chevrotain: 11.1.2 + cytoscape: 3.34.3 + cytoscape-cose-bilkent: 4.1.0(cytoscape@3.34.3) + cytoscape-fcose: 2.2.0(cytoscape@3.34.3) + d3: 7.9.0 + d3-sankey: 0.12.3 + dagre-d3-es: 7.0.14 + dayjs: 1.11.23 + dompurify: 3.4.15 + elkjs: 0.9.3 + es-toolkit: 1.52.0 + katex: 0.16.47 + khroma: 2.1.0 + marked: 16.4.2 + roughjs: 4.6.6 + stylis: 4.4.0 + ts-dedent: 2.3.0 + uuid: 14.0.2 + micromark-core-commonmark@2.0.3: dependencies: decode-named-character-reference: 1.3.0 @@ -5283,6 +6118,8 @@ snapshots: is-decimal: 2.0.1 is-hexadecimal: 2.0.1 + path-data-parser@0.1.0: {} + path-to-regexp@8.4.2: {} pathe@2.0.3: {} @@ -5337,6 +6174,13 @@ snapshots: pluralize@8.0.0: {} + points-on-curve@0.2.0: {} + + points-on-path@0.2.1: + dependencies: + path-data-parser: 0.1.0 + points-on-curve: 0.2.0 + postcss-load-config@6.0.1(jiti@2.7.0)(postcss@8.5.26)(yaml@2.9.0): dependencies: lilconfig: 3.1.3 @@ -5517,6 +6361,8 @@ snapshots: resolve-from@5.0.0: {} + robust-predicates@3.0.3: {} + rollup@4.63.0: dependencies: '@types/estree': 1.0.9 @@ -5549,8 +6395,19 @@ snapshots: '@rollup/rollup-win32-x64-msvc': 4.63.0 fsevents: 2.3.3 + roughjs@4.6.6: + dependencies: + hachure-fill: 0.5.2 + path-data-parser: 0.1.0 + points-on-curve: 0.2.0 + points-on-path: 0.2.1 + + rw@1.3.3: {} + safe-stable-stringify@2.5.0: {} + safer-buffer@2.1.2: {} + scheduler@0.27.0: {} scroll-into-view-if-needed@3.1.0: @@ -5658,6 +6515,8 @@ snapshots: client-only: 0.0.1 react: 19.2.8 + stylis@4.4.0: {} + sucrase@3.35.1: dependencies: '@jridgewell/gen-mapping': 0.3.13 @@ -5709,6 +6568,8 @@ snapshots: trough@2.2.0: {} + ts-dedent@2.3.0: {} + ts-interface-checker@0.1.13: {} tslib@2.8.1: {} @@ -5814,6 +6675,8 @@ snapshots: util-deprecate@1.0.2: {} + uuid@14.0.2: {} + vfile-message@4.0.3: dependencies: '@types/unist': 3.0.3 diff --git a/site/app/layout.tsx b/site/app/layout.tsx index f1b72ea..811b659 100644 --- a/site/app/layout.tsx +++ b/site/app/layout.tsx @@ -3,6 +3,7 @@ import { RootProvider } from "fumadocs-ui/provider"; import type { Metadata, Viewport } from "next"; import { Caveat, Inter, JetBrains_Mono, Space_Grotesk } from "next/font/google"; import type { ReactNode } from "react"; +import { SITE_URL } from "@/lib/site-url"; const inter = Inter({ subsets: ["latin"], variable: "--font-inter" }); const spaceGrotesk = Space_Grotesk({ subsets: ["latin"], variable: "--font-space-grotesk" }); @@ -13,7 +14,7 @@ const DESCRIPTION = "Turn an OpenAPI spec into safe, typed, human-reviewed WebMCP tools. Real files in your repo: contracts regenerate, your code survives, safety audit built in."; export const metadata: Metadata = { - metadataBase: new URL("https://webmcp-stack.vercel.app"), + metadataBase: new URL(SITE_URL), title: { template: "%s | webmcp-stack", default: "webmcp-stack: generate WebMCP tools from the API spec you already have", diff --git a/site/app/llms-full.txt/route.ts b/site/app/llms-full.txt/route.ts new file mode 100644 index 0000000..52f7288 --- /dev/null +++ b/site/app/llms-full.txt/route.ts @@ -0,0 +1,39 @@ +import { readFile } from "node:fs/promises"; +import { join } from "node:path"; + +/** Generated at build time, so it is a static file on the deployed site. */ +export const dynamic = "force-static"; + +/** + * Every docs page as raw markdown, in the order the navigation uses, for an + * agent that would rather read the whole thing in one request than crawl. + * Frontmatter is stripped; the page bodies already start with their headings. + */ +export async function GET() { + const dir = join(process.cwd(), "content/docs"); + const meta = JSON.parse(await readFile(join(dir, "meta.json"), "utf8")) as { + pages: string[]; + }; + + const parts: string[] = [ + "# webmcp-stack docs (full text)", + "", + "> The complete documentation for @webmcp-stack/codegen, concatenated for a single read.", + "", + ]; + + for (const slug of meta.pages) { + const raw = await readFile(join(dir, `${slug}.mdx`), "utf8"); + const body = raw + .replace(/^---[\s\S]*?---\s*/, "") + // Fumadocs code annotations are presentation only; a reader of the raw + // markdown should not see "// [!code highlight]". + .replace(/ ?\/\/ ?\[!code[^\]]*\]/g, "") + .trim(); + parts.push(body, "", "---", ""); + } + + return new Response(parts.join("\n"), { + headers: { "content-type": "text/plain; charset=utf-8" }, + }); +} diff --git a/site/app/llms.txt/route.ts b/site/app/llms.txt/route.ts new file mode 100644 index 0000000..b086d85 --- /dev/null +++ b/site/app/llms.txt/route.ts @@ -0,0 +1,40 @@ +import { SITE_URL } from "@/lib/site-url"; +import { source } from "@/lib/source"; + +/** Generated at build time, so it is a static file on the deployed site. */ +export const dynamic = "force-static"; + +const BASE = SITE_URL; + +/** + * The llms.txt index: a short, curated map of the docs for an agent to read + * before fetching anything. Format follows the llms.txt convention. + */ +export function GET() { + const pages = source.getPages().sort((a, b) => a.url.localeCompare(b.url)); + const lines = [ + "# webmcp-stack", + "", + "> Generate safe, reviewable WebMCP tools from the API contract you already have, so AI agents can act on a site in the signed-in user's session.", + "", + "## Docs", + "", + ...pages.map((page) => + `- [${page.data.title}](${BASE}${page.url}): ${page.data.description ?? ""}`.trim(), + ), + "", + "## Full text", + "", + `- [llms-full.txt](${BASE}/llms-full.txt): every docs page concatenated for a single read.`, + "", + "## Optional", + "", + "- [GitHub](https://github.com/SouravInsights/webmcp-stack)", + "- [npm](https://www.npmjs.com/package/@webmcp-stack/codegen)", + "", + ]; + + return new Response(lines.join("\n"), { + headers: { "content-type": "text/plain; charset=utf-8" }, + }); +} diff --git a/site/app/robots.ts b/site/app/robots.ts new file mode 100644 index 0000000..2f9fc3c --- /dev/null +++ b/site/app/robots.ts @@ -0,0 +1,15 @@ +import type { MetadataRoute } from "next"; +import { SITE_URL } from "@/lib/site-url"; + +const BASE = SITE_URL; + +/** + * Everything here is meant to be read, including by AI agents. The docs also + * publish /llms.txt and /llms-full.txt for exactly that. + */ +export default function robots(): MetadataRoute.Robots { + return { + rules: [{ userAgent: "*", allow: "/" }], + sitemap: `${BASE}/sitemap.xml`, + }; +} diff --git a/site/app/sitemap.ts b/site/app/sitemap.ts new file mode 100644 index 0000000..f4bb4ee --- /dev/null +++ b/site/app/sitemap.ts @@ -0,0 +1,15 @@ +import type { MetadataRoute } from "next"; +import { SITE_URL } from "@/lib/site-url"; +import { source } from "@/lib/source"; + +const BASE = SITE_URL; + +export default function sitemap(): MetadataRoute.Sitemap { + const pages = source.getPages().map((page) => ({ + url: `${BASE}${page.url}`, + changeFrequency: "monthly" as const, + priority: 0.6, + })); + + return [{ url: BASE, changeFrequency: "monthly", priority: 1 }, ...pages]; +} diff --git a/site/components/mdx.tsx b/site/components/mdx.tsx index 0b47c13..4404138 100644 --- a/site/components/mdx.tsx +++ b/site/components/mdx.tsx @@ -1,9 +1,27 @@ +import { Accordion, Accordions } from "fumadocs-ui/components/accordion"; +import { File, Files, Folder } from "fumadocs-ui/components/files"; +import { Step, Steps } from "fumadocs-ui/components/steps"; +import { Tab, Tabs } from "fumadocs-ui/components/tabs"; import defaultMdxComponents from "fumadocs-ui/mdx"; import type { MDXComponents } from "mdx/types"; +import { Mermaid } from "@/components/mermaid"; export function getMDXComponents(components?: MDXComponents) { return { ...defaultMdxComponents, + // Standard Fumadocs pieces, registered once so docs pages can use them + // without an import: Steps for procedures, Tabs for alternatives, + // Accordions for optional detail, Files for a directory tree. + Accordion, + Accordions, + File, + Files, + Folder, + Mermaid, + Step, + Steps, + Tab, + Tabs, ...components, } satisfies MDXComponents; } diff --git a/site/components/mermaid.tsx b/site/components/mermaid.tsx new file mode 100644 index 0000000..0565c55 --- /dev/null +++ b/site/components/mermaid.tsx @@ -0,0 +1,65 @@ +// biome-ignore-all lint/security/noDangerouslySetInnerHtml: this component renders Mermaid's SVG from this repo's own diagram source, not from user input. +"use client"; + +import { useTheme } from "next-themes"; +import { use, useEffect, useId, useState } from "react"; + +/** + * Renders a Mermaid diagram on the client. `remarkMdxMermaid` turns a + * ```mermaid code block in MDX into ``, so a diagram is + * authored as text and stays reviewable in the raw markdown (which also flows + * into llms-full.txt). + * + * This follows the Fumadocs recommended renderer. Mermaid is imported on + * demand, so pages without a diagram never load it. + */ +export function Mermaid({ chart }: { chart: string }) { + const [mounted, setMounted] = useState(false); + + useEffect(() => { + setMounted(true); + }, []); + + if (!mounted) return; + return ; +} + +const cache = new Map>(); + +function cachePromise(key: string, setPromise: () => Promise): Promise { + const cached = cache.get(key); + if (cached) return cached as Promise; + + const promise = setPromise(); + cache.set(key, promise); + return promise; +} + +function MermaidContent({ chart }: { chart: string }) { + const id = useId(); + const { resolvedTheme } = useTheme(); + const { default: mermaid } = use(cachePromise("mermaid", () => import("mermaid"))); + + mermaid.initialize({ + startOnLoad: false, + securityLevel: "loose", + fontFamily: "inherit", + themeCSS: "margin: 1.5rem auto 0;", + theme: resolvedTheme === "dark" ? "dark" : "default", + }); + + const { svg, bindFunctions } = use( + cachePromise(`${chart}-${resolvedTheme}`, () => { + return mermaid.render(id, chart.replaceAll("\\n", "\n")); + }), + ); + + return ( +
{ + if (container) bindFunctions?.(container); + }} + dangerouslySetInnerHTML={{ __html: svg }} + /> + ); +} diff --git a/site/content/docs/after-you-generate.mdx b/site/content/docs/after-you-generate.mdx new file mode 100644 index 0000000..6ee2034 --- /dev/null +++ b/site/content/docs/after-you-generate.mdx @@ -0,0 +1,162 @@ +--- +title: After you generate +description: The tools exist. Now decide what agents may actually do, check it, ship journeys for the real goals, and keep it honest as your API changes. +--- + +Generating is the easy part. This page is the rest: the loop you actually run, +in the order that works. + + + If you have not generated anything yet, start with the + [Quickstart](/docs/quickstart). This page assumes you have a `src/webmcp` + folder and a passing `generate`. + + +## 1. Look at what you got + +Before you change anything, read the report and the scorecard. + +```bash +npx @webmcp-stack/codegen dev # browse and inspect every tool +npx @webmcp-stack/codegen verify # the quality scorecard, exits 1 on errors +``` + +The dashboard is for understanding, not decorating: every tool with its route, +risk label, and findings. `verify` tells you where the surface is weak. Fix +errors first; they block generation. Warnings can wait until the tool matters. + +A generated project looks like this. One file per tool, plus the shared runtime, +the journey factory, and the barrel that registers everything: + + + + + + + + + + + + + + +## 2. Decide what agents may actually do + +Reads are registered so you can see the surface. Writes and destructive tools +are generated but **not registered**. Enabling one is a deliberate edit, in the +dashboard or in `.webmcp-codegen.json`: + +```json title=".webmcp-codegen.json" +{ + "overrides": { + "delete-trip": { "enabled": true } + } +} +``` + +Turn on only what you would let a careful assistant do, and turn them on one at +a time. A surface with four reviewed writes is worth more than one with forty +unreviewed ones. If an endpoint exists but an agent should never touch it, leave +it withheld or exclude it with `safety.exclude`. + +## 3. Fix the descriptions that matter + +This is the highest-leverage edit in the whole loop. A tool's name and +description are literally part of the prompt the model reasons over, and the +default from your spec is often a terse summary written for humans. + +Open the tools that matter and rewrite the description as three things: what it +does, when to use it, and what it returns. + +```text +Before: Get trips +After: List the trips the signed-in user has saved, newest first. Returns an + array of trips with title, dates, and place. +``` + +Descriptions live in the generated region, so edit them in +`.webmcp-codegen.json` (the dashboard writes here too), never in the file above +the marker. Your overrides survive every regeneration. + +## 4. Test it the way a person would + +Open your app in Chrome, turn on `chrome://flags/#enable-webmcp-testing`, and +use the DevTools panel under **Application > WebMCP**. Call a read tool and +check the data. Call a write tool and go all the way through the confirmation +dialog, because that is the path a real user takes. + +The dashboard can run tools too, but it runs them server-side, without your +browser session. Use it to check that the request is built correctly; use +DevTools to check it works while signed in. See [Chrome DevTools WebMCP +panel](/docs/devtools). + +## 5. Write journeys for the goals that take several calls + +Some things a user asks for cannot be one tool call. Booking a ride is resolve +the destination, set the pickup time, then confirm. That shape is a +[journey](/docs/journeys): a shared draft, one tool per step, and a submit gate +that refuses until every step is done and asks the human before the real write. + +Journeys are the part the generator cannot derive from your API, because the +flow lives in your product. Your coding agent writes them, guided by the skill +file. [Working with your coding agent](/docs/working-with-your-agent) shows how +to ask. + +## 6. Make the effects visible + +When a tool changes something, the page should move. An agent acting while a +person watches, with no visible change, is a bug in the experience even when the +request succeeded. Update the same UI your human flow updates: navigate, +invalidate a query, dispatch an event. + +The generated tools leave a marked spot for exactly this, below the marker where +you own the code. The pattern is in [Make the effect +visible](/docs/visible-effects). + +## 7. Put the checks in CI + +`verify` exits `1` on errors, so it is a one-line CI gate: + +```bash +npx @webmcp-stack/codegen verify +``` + +Run it on pull requests that touch your spec or your tools. A destructive +endpoint that lost its classification, a description that tries to instruct the +agent, a tool with nothing an agent can act on: all of it shows up before it +ships. Add `generate --dry-run` if you also want the audit for the spec itself. + +## 8. Regenerate as your API changes + +The point of generating is that the second run is safe. When your API changes, +re-run `generate`: + +```bash +npx @webmcp-stack/codegen generate +``` + +Only the generated region of each file updates. Your `execute()` bodies, your +overrides, and your dashboard edits stay put. If you hand-edited a generated +region, the tool leaves your file alone and writes its version to a `.new` +sibling instead of overwriting you. See [The regeneration +contract](/docs/regeneration). + +## 9. Keep the surface small + +The healthiest sign is not "we exposed everything". It is a surface an agent can +hold in its head. Three habits do most of the work: + +- Withhold anything an agent should not do on its own. +- Prefer a journey over exposing each step as a standalone tool. +- When the surface grows, ask whether two tools are really one intent. + +`verify` warns when the surface grows past what agents handle well, and counts +journey tools too, so this stays visible instead of drifting. + +## 10. When something breaks + +Most first-run problems are a URL or a session, not the tool. The +[Troubleshooting](/docs/troubleshooting) page covers the common ones: a 404 on a +relative URL, the dashboard working where the browser does not, and a +withheld tool that never appears. diff --git a/site/content/docs/cli.mdx b/site/content/docs/cli.mdx index 9cb1cf4..f6cd5d4 100644 --- a/site/content/docs/cli.mdx +++ b/site/content/docs/cli.mdx @@ -20,8 +20,6 @@ The main command. Resolves where tools come from and where they go, in this orde --force Write files even when the audit reports errors --config PATH Use a config file at PATH --watch Re-generate when files change - --llm Improve the names and descriptions of the tools being generated (LLM) - --suggest Find schemas worth declaring as tools (LLM) ``` A successful run also wires registration into your app (two additive lines; see @@ -42,40 +40,6 @@ Re-runs on every relevant file change. Regeneration is cheap and merge-safe, so keeps tools in sync while you edit the spec. The watcher ignores its own outputs (`src/webmcp`, `.webmcp-codegen.json`), so it never loops. -## The two LLM flags - -Both are **opt-in** — plain `generate` never touches a model, never sends anything anywhere, -never needs a key. Turn one on only when you want the LLM's help. They answer two different -questions: - -**`generate --llm`** — *"make the tools I already generate better."* -You have a spec (or a config) and tools generate. The LLM reads each tool's name, description, -and fields and proposes clearer descriptions and names, printed as `◦` draft lines for you to -review. Nothing is auto-applied; your hand-written text always wins; safety classification is -never the model's call. - -**`generate --suggest`** — *"find tools I haven't declared yet."* -You have validation schemas but no generated tools. The CLI finds your schema modules itself -(no file path needed) and the LLM proposes which schemas are worth declaring as tools. Nothing -is written; declaring is your edit to `codegen.config.mjs`. - -The rule of thumb: **tools already exist → `--llm`; starting from schemas → `--suggest`.** - -### The API key - -Neither flag requires your own key. The first time you run one without a key configured, an -interactive terminal offers three choices: - -- **Use the free hosted tier** — webmcp-stack's shared key (rate-limited), zero setup. The key - lives on our server, never in the package. -- **Enter my own API key** — OpenRouter, OpenAI, or any OpenAI-compatible provider. Used for - this command only, never stored, input is masked. -- **Skip** — continue without LLM features, exactly as if you'd passed neither flag. - -In CI (non-interactive), both flags quietly produce the deterministic output — a prompt never -blocks a pipeline, and a failing provider becomes one note line, never a failure. To use your -own key always, set `WEBMCP_LLM_API_KEY` or `OPENAI_API_KEY` in your environment. - ## `npx @webmcp-stack/codegen verify` Measures your generated tools against the quality standard they are supposed to meet, before @@ -83,11 +47,11 @@ anything ships. Runs the same pipeline as `generate` (writing nothing) and repor scorecard: ``` -✓ Names: all 35 names within 30 characters, verb-first -! Descriptions: 2 problems -✓ Fields: every input field described, nested ones included -✓ Annotations: reads declare readOnlyHint; content declares its trust -! Surface: 35 registered +ok Names: all 35 names within 30 characters, verb-first +! Descriptions: 2 problems +ok Fields: every input field described, nested ones included +ok Annotations: reads declare readOnlyHint; content declares its trust +! Surface: 35 registered ``` Each finding names the tool and what to do about it. Exits 1 on error-level findings (a tool diff --git a/site/content/docs/devtools.mdx b/site/content/docs/devtools.mdx index 44755fd..c7ad637 100644 --- a/site/content/docs/devtools.mdx +++ b/site/content/docs/devtools.mdx @@ -1,12 +1,13 @@ --- title: Chrome DevTools WebMCP panel description: Inspect, test, and debug your tools with Chrome's built-in WebMCP panel. +--- Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exposes to AI agents. It's the fastest way to verify your tools work before an agent ever sees them. ## Open the panel -1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 146+) +1. Enable WebMCP: `chrome://flags/#enable-webmcp-testing` (Chrome 149+, Edge 150+. This flag is for local development; production pages use the origin trial.) 2. Open your app and open DevTools (F12) 3. Click the **Application** tab 4. In the sidebar, click **WebMCP** @@ -15,8 +16,8 @@ Chrome DevTools has a dedicated WebMCP panel that shows every tool your page exp The panel has two sections: -- **Available Tools** — every tool currently registered on the page, with name, description, and an invocation counter -- **Invoked Tools** — a log of every tool call, with status, input, and output +- **Available Tools**: every tool currently registered on the page, with name, description, and an invocation counter +- **Invoked Tools**: a log of every tool call, with status, input, and output ## Test a tool manually diff --git a/site/content/docs/guides.mdx b/site/content/docs/guides.mdx index 3f99642..ee05be0 100644 --- a/site/content/docs/guides.mdx +++ b/site/content/docs/guides.mdx @@ -43,7 +43,7 @@ actually use. Use `safety.exclude` in the config for whole families (`["internal The generated tools register on page load. To watch that happen: -- **Chrome:** turn on `chrome://flags/#enable-webmcp-testing` (Chrome 146+) and reload. +- **Chrome:** turn on `chrome://flags/#enable-webmcp-testing` (Chrome 149+; production sites join the WebMCP origin trial) and reload. The *Model Context Tool Inspector* extension shows the registered tools and lets you invoke them by hand. - **Other browsers:** add the WebMCP polyfill to your page; the same registration code @@ -52,20 +52,6 @@ The generated tools register on page load. To watch that happen: dashboard (server-side, so without your browser session; auth'd endpoints will answer accordingly). -## The dashboard - -```bash -npx @webmcp-stack/codegen dev -``` - -![The webmcp-codegen dashboard](/images/dev-server-dashboard.png) - -A local control panel for the generated tools: browse and search them grouped by risk -level, see each one's route and audit findings, edit descriptions, toggle tools on and -off, and run them directly. Edits are written to `.webmcp-codegen.json` as overrides, so -they survive regeneration and can be committed. The dashboard never touches your app: it -is a localhost page served by the CLI, gone when you Ctrl+C. - ## CI The audit exits non-zero on errors, which makes it a CI check: diff --git a/site/content/docs/index.mdx b/site/content/docs/index.mdx index b500390..8b85d9e 100644 --- a/site/content/docs/index.mdx +++ b/site/content/docs/index.mdx @@ -1,80 +1,94 @@ --- title: Introduction -description: Generate safe, typed, human-reviewed WebMCP tools from your OpenAPI spec or validation schemas. +description: Generate safe, reviewable WebMCP tools from the API contract you already have, so agents can act on your site without guessing. --- ## The problem -WebMCP lets your web app expose tools that AI agents can call, but somebody has to write -those tools. Today that means hand-writing a `registerTool()` call per action: a name, a -description, a JSON Schema for the inputs, an `execute` function. For one endpoint that's a -chore. For an API with seventy endpoints it's a project nobody starts. +People are already pointing agents at the sites they use. Today those agents +read your HTML and guess which control does what. [WebMCP](https://github.com/webmachinelearning/webmcp) +is the browser's answer: your page can register a small set of typed tools that +an agent calls directly, in the signed-in user's session. -And the details matter more than they look: a tool's **description is literally part of the -prompt** the agent reasons over, and its **schema is the contract** the agent fills in. -Hand-written copies of things your code already knows go stale, drift, and lie. +Somebody has to write those tools. Hand-writing a `registerTool()` call per +action, a name, a description, a JSON Schema, an `execute` function, is a chore +for one endpoint and a project nobody starts for seventy. And the details matter +more than they look: a tool's **description is part of the prompt** the agent +reasons over, and its **schema is the contract** the agent fills in. Hand-written +copies of what your code already knows drift, and drift means the agent calls +the wrong thing. -## What webmcp-codegen does +## What webmcp-stack does -You already have the source of truth: an OpenAPI spec, or validation schemas (zod, valibot, -arktype, TypeBox) your app uses. webmcp-codegen turns them into WebMCP tools, written as real -TypeScript files in your repo: +You already have the source of truth: an OpenAPI spec, or the validation +schemas (zod, valibot, arktype, TypeBox) your app uses. `@webmcp-stack/codegen` +turns them into WebMCP tools, written as real TypeScript files in your repo: ```bash npx @webmcp-stack/codegen generate ``` -``` -Detected apps/server/openapi/openapi.json (override with --spec) -Found your web app: apps/web (next) - -webmcp-codegen: 7 tool(s) +In that one run, three things happen. Tools are generated: reads work +immediately, mutations are generated but withheld until you enable them. Bad +agent endpoints are filtered or flagged: webhooks, auth, admin. And registration +is wired into your app, as two additive lines you can see and undo. - note: Stripped the shared "v1" version prefix from tool names. +Beyond the endpoints: - 1 endpoint(s) skipped: - POST /v1/payments/webhook: a webhook receives server callbacks - - list-trips [read] ← GET /v1/trips/ - create-trip [write] withheld - delete-trip [destructive] withheld - ... - - registration: - created apps/web/src/webmcp/register.tsx - added 2 lines to apps/web/src/app/layout.tsx -``` - -Three things happen in that one run. Tools are generated (read tools with working -implementations, mutations generated but disabled until you uncomment them). Endpoints that -should not be agent tools are filtered or flagged (webhooks, auth, admin). And registration -is wired into your app, two additive lines you can see and undo. +- **Handshake grouping.** Two calls that are really one action (request an + upload, then complete it) become one withheld coarse tool. +- **Journeys.** Multi-step goals get a shared draft, one tool per step, and a + submit gate that asks the human before the real write. See + [Journeys](/docs/journeys). +- **A skill file.** `generate` writes `.agents/skills/webmcp-tools/SKILL.md` so + your own coding agent knows the rules when it extends the surface. See + [Working with your coding agent](/docs/working-with-your-agent). ## Three promises -**Tools work out of the box.** The spec knows the method, path, and parameters, so read -tools are born calling your real API with the signed-in user's session. Mutations get the -same working code, commented out, one deliberate edit away. - -**You own the output.** Generated files are plain TypeScript in your repo. No runtime -dependency on webmcp-codegen. Uninstall it after generating and everything still works. - -**Regeneration never clobbers your code.** The API contract lives above a marker line and -regenerates freely. Your code lives below it and is never touched. Hand-edit the generated -region and you get a `.new` file to merge, never a silent overwrite. - -## See it before you run it - -Every claim on this page is verifiable without installing anything: - -```bash -npx @webmcp-stack/codegen generate --dry-run -``` - -A dry run writes nothing. It shows you the tools, the safety report, and the plan. +**Tools work out of the box.** The spec knows the method, path, and parameters, +so read tools are born calling your real API with the signed-in user's session. +Mutations get the same working code, commented out, one deliberate edit away. + +**You own the output.** Generated files are plain TypeScript in your repo. No +runtime dependency on webmcp-stack. Uninstall it after generating and everything +still works. + +**Regeneration never clobbers your code.** The API contract lives above a marker +line and regenerates freely. Your code lives below it and is never touched. +Hand-edit the generated region and you get a `.new` file to merge, never a +silent overwrite. + +## What it does not do + +Being clear about the edges is part of using it well: + +- It does not work without a contract you maintain. It generates from an + OpenAPI spec or the validation schemas your app already uses, not by scanning + your code, and a vague contract makes vague tools. This is on purpose. See + [Who this is for](/docs/why-agent-tools#who-this-is-for). +- It does not decide your product's intents. That is your knowledge, or your + agent's, guided by the skill file. +- It does not make an unsafe API safe. Every tool calls your real endpoint, so + your server still validates, authorizes, and rate-limits. +- It does not run a model. Generation is deterministic, no key, no network. + +## Where to start + + + + Why a small, well-described surface matters, and what a model can and cannot do with your site. + + + One command, working tools, wired into your app. + + + The loop that turns a generated surface into one you trust. + + ## Status -WebMCP is an early-stage spec. The tools work today in Chrome behind a flag, and elsewhere -with a small polyfill. OpenAPI is the supported source; tRPC, Zod, and Prisma are on the -roadmap. +WebMCP is an early-stage spec. The tools work today in Chrome behind a flag and +in the origin trial, and elsewhere with a small polyfill. OpenAPI is the +supported source; validation schemas are supported; tRPC is on the roadmap. diff --git a/site/content/docs/journeys-faq.mdx b/site/content/docs/journeys-faq.mdx new file mode 100644 index 0000000..96f9560 --- /dev/null +++ b/site/content/docs/journeys-faq.mdx @@ -0,0 +1,170 @@ +--- +title: Journey questions, answered +description: The draft, the lifecycle, what the agent does and what you do, and the edge cases, answered in plain terms. +--- + +The [journeys page](/docs/journeys) explains the feature. This page answers the +questions people ask once they read the code. None of them are dumb questions; +they are the ones almost everyone has. + +## Is `journey.webmcp.ts` a journey? + +No, and this is the most common confusion. That file never creates a journey. It +is the factory: a reusable `createJourney()` function that journey files call. +Think of a Lego baseplate and the models built on it. `journey.webmcp.ts` is the +baseplate, shared and owned by the generator. `book-ride.webmcp.ts` is a model +someone builds on it. The engine lives in one file; the journeys live elsewhere +and are written per product. + +## What are the three parts, in plain terms? + +1. **The draft** is one plain object in memory, created when the page loads. + Steps write into it; the submit reads everything out of it. +2. **Step tools** are ordinary WebMCP tools, one per step + (`book-ride-resolve-destination`). When the agent calls one, it does the + step's work, stores the result on the draft, and replies "Stored. Still + needed: X" or "Stored. The journey is ready." That reply is how the agent + knows what to do next. There is no hidden orchestration; the agent works one + call at a time and reads the messages. +3. **The submit gate** is one final tool (`book-ride-submit`) that does four + things in order: check nothing is missing and refuse if it is, ask the human + to confirm, run the real write with the finished draft, and clear the draft. + +## Why is it called a "draft"? + +Same reason as a draft email. It is the version in progress, built up piece by +piece, and nothing is sent until the end. It is a draft because it is *not* the +real thing: nothing reaches your backend from it until the submit runs. + +## What does "storing something in the draft" actually mean? + +Adding a labeled value to that one object, like `destination: {...}` or +`pickupAt: "2026-03-14T19:30:00Z"`. It is like writing another line on a sticky +note. There is no database, no file, and no network call in the storing itself. + +## What does "in memory" mean if I am not technical? + +Picture a whiteboard in a room versus a filing cabinet down the hall. The +whiteboard is right there, fast, and everyone in the room can read it, but the +moment the room closes it is gone. The filing cabinet keeps things after +everyone leaves. The draft is the whiteboard: it lives only inside the browser +tab's running memory. Close the tab, refresh, or navigate away and it is wiped. +Nothing durable, ever. + +## What is the shape of the draft? + +The simplest possible: a flat bag of labeled values, for example +`{ destination: {...}, pickupAt: "2026-03-14T19:30:00Z" }`. Nothing enforces +nesting, and any step can write any field. + +## Will it hold data reliably for a session? + +It depends what you mean by session: + +- **Within one continuous page view** (tab open, agent working, no refresh): yes, + completely. It is a variable in memory and nothing drops it mid-flow. +- **Across a reload, a closed tab, or a new tab:** no, deliberately. It is never + meant to outlive one uninterrupted visit. + +## How does it store results from the real backend? + +Not every step touches the backend. A step like `set-pickup-time` has no logic; +it stores whatever the agent passed in, verbatim, with no network call. A step +like `resolve-destination` calls the app's real geocoder and stores the real +answer. So the draft ends up holding a mix: some fields are "what the agent +said", some are "what the server said". Either way, nothing is created or +changed for real until the submit. + +## Why is the draft created only once per page load? + +Because `createJourney()`, the function that sets up the draft, runs once at +page load, not per attempt. There is one shared scratchpad per journey per page +visit. That is fine for the realistic case: one agent, one user, one thing at a +time. The unhandled edge case is two parallel attempts at the *same* journey on +one page, which would share and overwrite the same draft. Accepted, not handled. + +## What is a "half-finished journey", and why would it happen? + +The agent resolves the destination, then the person closes the laptop or the tab +crashes, before the rest of the steps and the submit. Some steps done, nothing +sent. Keeping the draft in memory means a reload always starts clean, with no +stale, half-filled attempt from three days ago reappearing with an outdated +result. Throwing away incomplete progress is the feature, not a gap. + +## Does a journey reuse the tools codegen already generated? + +Yes, directly, and it never reimplements anything. A tool-backed step calls the +same underlying request the standalone tool uses, and the submit's `run` is the +existing write tool's execute. The journey wraps them with the gate and the +confirmation. Bonus: a risky write like `request-ride`, normally generated +withheld, never has to be enabled as a standalone tool at all. The journey's +submit becomes its only door, with the prerequisite checks and a human +confirmation built in. Wrapping a risky tool in a journey is a safer way to +expose it than enabling it directly. + +## How does the agent know these tools exist? + +There is no special channel: ordinary WebMCP discovery. On page load, every +registered tool, including journey steps and submits, lands in one flat list the +browser exposes. The agent asks the page what it can do and gets the whole list, +with no formal distinction between a normal tool and part of a journey. The only +grouping signal is in the text: each step's name carries the journey prefix +(`book-ride-...`), and each description ends with something like *"Part of +book-ride: Book a ride to a destination the user gives."* The agent reads that +and infers the connection, the same way it figures out anything else. The +factory stitches it mechanically, so it cannot be forgotten. + +## What does the agent do, and what do I do? + +The agent's job is orchestration and conversation: work out what to call next +from the replies, and ask the user for anything it cannot get from the backend. +Your job is exactly one thing: the final yes or no on the confirmation dialog +before anything real happens. The agent cannot click that dialog for you. A +human click is the only way through. + +## Can the agent call the steps out of order? + +The order is not enforced by code, so an agent could call `set-pickup-time` +before `resolve-destination`. It mostly self-corrects: a step that needs an +earlier step's draft field fails with a readable message naming what is missing, +which nudges the agent back. One honest caveat: that self-correction depends on +the step failing *usefully* when called early, which is on whoever writes the +journey, not something the shared machinery guarantees. + +## What happens if I decline the confirmation? + +The agent is told "The user declined this action", and the draft is left +untouched, not wiped. You can change your mind, adjust something, and submit +later without starting over. + +## Why can't codegen just generate the journeys for me? + +Because the flow is not in your API spec. An OpenAPI file says `POST /v1/rides` +exists; it does not say that booking a ride is really resolve the destination, +then set the pickup time, then confirm. It also cannot know that your product +wants a person to approve the ride rather than let an agent request it +directly. That knowledge lives in the product. A CLI guessing flows produces +plausible-sounding garbage, tools that pass every mechanical check and are still +wrong. So codegen stops at a hint in its report and the bundled skill file +teaches your own coding agent how to write the journey file well. Your agent can +ask you about the product; the CLI cannot. + +## Won't journeys make my tool list even bigger? + +They can, if you let them. `verify` counts journey tools in the surface total, +one per step plus the submit gate, and warns when the whole surface grows past +what is good for agents. It is a warning, not a failed build: the browser has no +limit on how many tools you register, only agent quality does. + +Keeping the surface small is a convention you follow, not something the +generator does for you. A read tool that a journey uses still registers on its +own, so if it only makes sense inside the flow, withhold it in +`.webmcp-codegen.json` or exclude it with `safety.exclude`. Writes reachable +through a journey submit stay withheld as standalone tools unless you enable +them, so leave them withheld and the submit is their only door. + +## What if two steps use the same draft field name? + +The second overwrites the first, silently. The draft is a flat bag with no +protection, so give each step's fields distinct names when you write the +journey. diff --git a/site/content/docs/journeys.mdx b/site/content/docs/journeys.mdx new file mode 100644 index 0000000..a701345 --- /dev/null +++ b/site/content/docs/journeys.mdx @@ -0,0 +1,229 @@ +--- +title: Journeys +description: A few steps that share state, ending in one write a person confirms. Journeys are how you describe a goal that takes more than one tool call. +--- + +## When one tool is not enough + +Most tools do one thing and forget it: `list-trips`, `cancel-order`, +`search-products`. Some goals do not fit that shape. Booking a ride is not one +call, it is: + +1. turn the destination the user said into the coordinates the API needs, +2. ask when they want to be picked up, +3. actually request the ride, after the person agrees to it. + +Three things make this a different kind of problem: + +- **The steps depend on each other.** The ride request needs what the first two + steps produced. +- **The agent cannot invent a step's output.** It knows "SFO airport" as words. + The ride API needs coordinates, and those have to come from a real geocoder. + Ask a model to make them up and your server rejects the request. +- **The last step changes something real.** Requesting a ride should not happen + because an agent decided it should. A person should say yes. + +A **journey** is how you describe that goal once: a shared place to collect the +answers, one tool per step, and a submit that a human confirms. + +## The mental model: a form, filled in steps + +The easiest way to hold journeys in your head is a form. + +- **The draft is the form.** One object in the page's memory, for the life of + the page. Steps write on it; the submit reads from it. It is not a database + and nothing is saved to disk. +- **Each step fills in part of the form.** A step either calls one of your + existing tools and stores what it returns (the destination's coordinates), or + just takes the agent's input and stores it (the pickup time). Every step + replies with what is still blank, so the agent always knows the next move. +- **The submit is the Send button.** It stays disabled until every field is + filled, it shows the person what is about to happen, and only then does it run + the real tool. + +From the agent's side there is no hidden machinery. It sees a small flat list of +tools and fills them in one at a time, reading the "still needed" reply after +each call. Journeys do not coordinate the agent. They give it a form. + +## The whole thing in one example + +A rideshare app has two generated tools: `geocode-address` (a read) and +`request-ride` (a write, so it is withheld until enabled). The journey that ties +them into one goal is a single file: + +```ts +// src/webmcp/journeys/book-ride.webmcp.ts +import { createJourney } from "../journey.webmcp"; +import { geocodeAddressTool, fetchGeocodeAddress } from "../geocode-address.webmcp"; +import { executeRequestRide, type RequestRideInput } from "../request-ride.webmcp"; + +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", + + steps: { + // A tool-backed step. The agent gives an address as text; the real + // geocoder turns it into coordinates, and the step stores the result. + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: (place) => ({ destination: place }), // [!code highlight] + provides: ["destination"], // [!code highlight] + }, + + // A free step. No API call: the pickup time the agent passed in is + // stored on the draft as-is. + "set-pickup-time": { + description: "Set when the user wants to be picked up.", + input: { + type: "object", + properties: { + pickupAt: { type: "string", description: "The pickup time, in ISO 8601." }, + }, + required: ["pickupAt"], + }, + provides: ["pickupAt"], // [!code highlight] + }, + }, + + submit: { + description: "Request the ride and show the driver's details.", + // The draft becomes the real tool's input; the write runs after the human confirms. + build: (draft) => draft as RequestRideInput, // [!code highlight] + run: executeRequestRide, + }, +}); +``` + +That is the whole feature. There is no config block and no new language: a +journey is TypeScript that imports the tools you already generated. + +## What the agent sees + +On page load, `createJourney` registers three ordinary tools. They look like any +other tool; nothing marks them as special to the agent. + +```text +book-ride-resolve-destination Fills the destination. Part of "book-ride": Book a ride... +book-ride-set-pickup-time Set when the user wants to be picked up. Part of "book-ride"... +book-ride-submit Request the ride and show the driver's details. +``` + +The agent fills the form by calling them in order, reading each reply. The +whole exchange, as a diagram: + +```mermaid +sequenceDiagram + actor User + actor Agent as Your coding agent + participant Page as Page (WebMCP) + participant API as Your backend + + Note over Page: createJourney().register() ran on load + Page->>Agent: tools registered: resolve-destination, set-pickup-time, submit + + Agent->>Page: book-ride-resolve-destination({ address: "SFO airport" }) + Page->>API: geocode-address + API-->>Page: coordinates + Note right of Page: stored on the draft + Page-->>Agent: Stored. Still needed: set-pickup-time, submit + + Agent->>Page: book-ride-set-pickup-time({ pickupAt: "2026-03-14T19:30:00Z" }) + Page-->>Agent: Stored. Ready. Call book-ride-submit. + + Agent->>Page: book-ride-submit + Page->>User: confirm "Request the ride?" + User-->>Page: approves + Page->>API: request-ride(build(draft)) + API-->>Page: ride requested + Page-->>Agent: done + Note over Page: draft cleared +``` + +Two details are doing a lot of work here. The "still needed" replies are how the +agent learns the order without being told it in prose. And the submit cannot run +until the draft is full, so an agent that skips a step is refused with the list +of what is missing, not a failed request. + +## How state moves between steps + +Three fields move the data: + +- **`call`** does the work. For a tool-backed step it calls the raw caller your + generated file exports (`fetchGeocodeAddress`), not the agent-facing wrapper. + You reuse one request, and the schema stays single-sourced. +- **`store`** decides what lands on the draft. It receives the call's result and + returns the fields to save. This is where a resolved object becomes a named + form field. +- **`provides`** names the draft fields the step is responsible for. The submit + refuses until every promise in every step is on the draft. + +The submit's **`build`** is the reverse: it takes the finished draft and +assembles the real tool's input. Because the only path to the real input is +`build(draft)`, an agent cannot call the write with a half-filled form. + +## Rules worth knowing + +- **Keep journeys short.** Two to five steps. Past five, you are usually + describing two journeys. +- **Steps read, submit writes.** A step should call a read tool or fill the + draft. Put anything that changes real data in the submit, where the human is + asked first. `verify` marks a step read-only only when the tool it composes is + a read, and it cannot inspect code you write by hand. +- **Reuse, do not re-implement.** A tool-backed step inherits the generated + tool's description and schema and calls its raw caller. Edit the tool, + regenerate, and the journey follows. Never write a direct `fetch` in a journey + file; `verify` flags it. +- **Do not double-register.** A tool that only makes sense inside a flow should + not also register standalone. Withhold it in `.webmcp-codegen.json`, or exclude + it, so the journey is its only door. Journeys should shrink the surface, not + grow it. +- **The draft is per page load, per journey.** It is reliable within one page + view and gone on reload, on purpose. Two parallel attempts at the same journey + on one page share a draft; in practice agents work one thing at a time. + +The gate itself cannot be edited away. `journey.webmcp.ts` owns the confirmation +and the draft clearing, and it is regenerated on every `generate`. The step +logic is your code, which is why the rules above exist. + +## What ships, and what you write + +`generate` writes two things into your tools directory: + +- **`journey.webmcp.ts`**, the factory that defines the draft, registers the + steps, and holds the submit gate. Generator-owned, regenerated every run. +- **`index.ts`** imports every file in `journeys/` and registers whatever + `createJourney` returns. Drop a journey file in, re-run `generate`, and it is + live. + +What it never writes is the journey itself, because an OpenAPI spec cannot tell +it that "book a ride" is really *resolve, then schedule, then confirm*. That is +product knowledge. You write it, or your coding agent does, guided by the +[skill file](/docs/working-with-your-agent). + +## Journeys are not grouping + +Two features sound similar and are not: + +- **Grouping** merges endpoints that are one action split across two calls (a + request-upload plus a complete-upload) into a single tool. It happens at + generate time, straight from the spec, with no human input. +- **Journeys** chain different decisions that share state and end in one guarded + submit. They are written per product, after generation. + +Same action split across calls: group it. Several decisions along the way: a +journey. + +## Next + + + + How to prompt an agent to write a journey for your product. + + + The draft, the lifecycle, and the edge cases, in plain terms. + + + Where journeys fit in the wider loop of shipping a safe surface. + + diff --git a/site/content/docs/meta.json b/site/content/docs/meta.json index 0058791..72d7cea 100644 --- a/site/content/docs/meta.json +++ b/site/content/docs/meta.json @@ -2,13 +2,21 @@ "title": "webmcp-stack", "pages": [ "index", + "why-agent-tools", "quickstart", - "cli", - "configuration", + "after-you-generate", + "tool-or-journey", + "journeys", + "journeys-faq", + "working-with-your-agent", + "prompt-cookbook", + "testing", "safety", "regeneration", "visible-effects", "guides", + "configuration", + "cli", "devtools", "troubleshooting" ] diff --git a/site/content/docs/prompt-cookbook.mdx b/site/content/docs/prompt-cookbook.mdx new file mode 100644 index 0000000..eb5d2af --- /dev/null +++ b/site/content/docs/prompt-cookbook.mdx @@ -0,0 +1,145 @@ +--- +title: Prompt cookbook +description: Copy-paste prompts for the jobs that come up again and again on an agent surface, with what to check afterwards. +--- + +Every `generate` run writes `.agents/skills/webmcp-tools/SKILL.md`, so a coding +agent that reads your repo already knows the naming rules, the budgets, and the +journey pattern. What it cannot know is your product. These prompts give it that +context. + +Copy one, replace the names in it with yours, and run it. You do not need a +coding agent to use them; each one also works as a checklist for doing the task +by hand. + + + Read the diff before you accept it. The + [Working with your coding agent](/docs/working-with-your-agent) page covers + what to check, and why a write is always your decision, not the agent's. + + +## Improve one description + +Descriptions are the highest-leverage edit on the surface, because the model +reads them as part of the prompt. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and src/webmcp/list-trips.webmcp.ts. +The description reads mechanical. Rewrite it so an agent knows what the tool +does, when to use it, and what it returns, within the skill's budget. Put the +new text in .webmcp-codegen.json, not inside the generated region. +``` + +**Then:** run `verify`, and check that the override landed in +`.webmcp-codegen.json` rather than in the generated file. + +## Write a journey + +A good journey prompt names the goal, the value the agent cannot invent, and the +step that needs a human. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and the generated tools in src/webmcp. + +Write a journey called book-ride in src/webmcp/journeys. Goal: let a user book a +ride to a destination they give. + +- The ride API needs coordinates, which only the geocode endpoint can provide. + Compose the generated geocode-address tool for that step and store the result + on the draft as destination. +- The user sets a pickup time in a second step. +- Requesting the ride is a write, so it goes through the submit gate, not a + step. Use the real request-ride execute as the submit's run. + +Run npx @webmcp-stack/codegen verify when you are done and fix what it reports. +``` + +**Then:** confirm every step is a read, and the write sits in `submit`. + +## Review the whole surface + +Useful before a release or after a big spec change. + +```text +Read .agents/skills/webmcp-tools/SKILL.md and every file in src/webmcp. For each +tool, answer: is the name intent-shaped, does the description say what the tool +returns, and should an agent be able to call it at all? Do not enable any +withheld tool. Return a table with the tool, the verdict, and the reason. +``` + +**Then:** turn the table into issues or edits. An agent suggesting a write be +enabled is a question for you, not an action. + +## Name a tool or journey + +Naming is where an agent without product context goes wrong most quietly. + +```text +Before naming anything, read what this product is: [one or two sentences about +who uses it and when]. Then suggest names for [the tool or flow] that say what +the user is doing, not what the endpoint is. Give me three options and the +reasoning for each. +``` + +**Then:** pick the name yourself. A name can pass every mechanical check and +still be wrong for the product. + +## Fix a failing verify + +```text +Run npx @webmcp-stack/codegen verify and fix every error. For warnings, list +them and propose a fix for each; do not change a description's meaning just to +make it shorter. Re-run verify when you are done. +``` + +**Then:** read the warnings yourself. The trim is a judgment call, not a +mechanical one. + +## Do a safety pass + +```text +Read .agents/skills/webmcp-tools/SKILL.md. Review src/webmcp for three things: +output fields that look like personal data, descriptions that instruct the agent +instead of describing the tool, and read tools that actually perform writes. +Report findings with the file and line, and change nothing. +``` + +**Then:** decide what to fix. The report is the deliverable. + +## Add a tool that is not in the spec + +```text +I want agents to be able to [action]. Look at the tools in src/webmcp and the +source in codegen.config.mjs. Add a tool for it in the style the skill file +expects, or tell me if it should be a journey instead. Do not enable any write. +``` + +**Then:** check the schema and description against the skill's rules, and run +`verify`. + +## Enable a withheld write + +Do this one yourself, or have the agent prepare it while you approve. + +```text +I want to allow agents to [action]. Find the tool, show me what it does and +whether it confirms with the user before running, then enable that one tool in +.webmcp-codegen.json. Do not enable anything else. +``` + +**Then:** read the diff, then test the tool in the browser and go through the +confirmation dialog yourself. + +## Next + + + + What the skill file teaches, and how to check an agent's work. + + + Which shape a given problem should take. + + + How to test the generated code and a journey's logic. + + diff --git a/site/content/docs/quickstart.mdx b/site/content/docs/quickstart.mdx index 0d1452c..bbe140c 100644 --- a/site/content/docs/quickstart.mdx +++ b/site/content/docs/quickstart.mdx @@ -37,9 +37,19 @@ npx @webmcp-stack/codegen generate --dry-run Generated files do nothing until your app registers them once at startup. The CLI does this edit for you: -- **Next.js (app router):** creates `src/webmcp/register.tsx` (a small client component) - and adds an import plus `` to your root layout. -- **Vite + React:** adds an import and one `registerAllTools()` call to `main.tsx`. + + + +Creates `src/webmcp/register.tsx`, a small client component, and adds an import plus +`` to your root layout. + + + + +Adds an import and one `registerAllTools()` call to `main.tsx`. + + + Edits are additive only, idempotent, and printed in the report with undo instructions. If the CLI can't find your entry file confidently, it prints the two lines instead of @@ -49,7 +59,7 @@ guessing. 1. Start your app and open it in Chrome. 2. Turn on `chrome://flags/#enable-webmcp-testing` and reload. -3. Open DevTools → **Application** → **WebMCP** to see your tools listed. +3. Open DevTools, then **Application**, then **WebMCP**, to see your tools listed. 4. Click a tool to test it manually, or ask the agent to use one. The CLI suggests one, drawn from your spec ("list my trips"). @@ -76,3 +86,19 @@ the marker never changes; your code below it is never touched. Add summaries to your OpenAPI operations as you go. A tool's description goes straight into the agent's prompt, and the audit reminds you when one is missing. + +## 6. Keep going + +Generation is the start of the loop, not the end. Where to read next: + + + + Review the surface, enable writes deliberately, add the checks to CI, and regenerate as your API changes. + + + How to prompt your own agent to improve descriptions and write journeys. + + + The multi-step primitive for goals that take more than one call. + + diff --git a/site/content/docs/regeneration.mdx b/site/content/docs/regeneration.mdx index 58c7db9..c050af8 100644 --- a/site/content/docs/regeneration.mdx +++ b/site/content/docs/regeneration.mdx @@ -20,12 +20,12 @@ to be re-run every time your API changes. Each tool file has two regions, split by a marker comment: ```ts title="src/webmcp/delete-pet.webmcp.ts" -// ─── webmcp-codegen: generated. Do not edit this region. ─── +// --- webmcp-codegen: generated. Do not edit this region. --- // name, description, input schema, input type, hints, register() // ... everything derived from the API contract -// ─── webmcp-codegen: end generated. Your code below survives regeneration. ─── +// --- webmcp-codegen: end generated. Your code below survives regeneration. --- export async function executeDeletePet(input: DeletePetInput) { // This tool is withheld: nothing registers it, so agents cannot see or call it. diff --git a/site/content/docs/testing.mdx b/site/content/docs/testing.mdx new file mode 100644 index 0000000..4a6ceb0 --- /dev/null +++ b/site/content/docs/testing.mdx @@ -0,0 +1,201 @@ +--- +title: Testing your tools +description: Generated tools are ordinary TypeScript. Here is how to test them without a browser, and the one thing that still needs a browser. +--- + +The generated files are plain TypeScript in your repo, and they export the +pieces you want to test: a raw caller (`fetchGetTrip`) and an agent-facing +function (`executeGetTrip`). That means most of your surface can be tested the +way you test the rest of your app. + + + A journey's draft and submit gate are library code you did not write and do + not own. Test the `store` and `build` functions you wrote, and test the whole + flow in a browser. Do not test the generated region. + + + + +### Test the request without a browser + +`fetchGetTrip` calls your API through `callApi`, which uses `fetch`. Mock +`fetch` and assert both the request and the parsed body. This is the test that +catches a wrong path, method, or body long before an agent does. + +```ts title="src/webmcp/get-trip.test.ts" +// @vitest-environment happy-dom +import { afterEach, expect, it, vi } from "vitest"; +import { fetchGetTrip } from "./get-trip.webmcp"; + +afterEach(() => vi.unstubAllGlobals()); + +it("calls the endpoint and returns the parsed body", async () => { + const calls: string[] = []; + vi.stubGlobal( + "fetch", + vi.fn(async (input: RequestInfo) => { + calls.push(String(input)); + return new Response(JSON.stringify({ id: "trip_1", title: "Lisbon" }), { + status: 200, + headers: { "content-type": "application/json" }, + }); + }), + ); + + const trip = await fetchGetTrip({ id: "trip_1" }); + + expect(calls[0]).toContain("/v1/trips/trip_1"); + expect(trip).toMatchObject({ id: "trip_1" }); +}); +``` + +The `happy-dom` environment matters: `callApi` resolves a relative path against +`window.location.origin`. + + + +### Test the result the agent reads + +`executeGetTrip` wraps the response with `toolResult`, which is the text an +agent actually sees. Assert the shape, not the whole string. + +```ts +import { executeGetTrip } from "./get-trip.webmcp"; + +it("wraps the result for the agent", async () => { + vi.stubGlobal( + "fetch", + vi.fn(async () => new Response(JSON.stringify({ id: "trip_1" }), { status: 200 })), + ); + + const result = await executeGetTrip({ id: "trip_1" }); + + expect(result.isError).toBeUndefined(); + expect(result.content[0]?.text).toContain("trip_1"); +}); +``` + + + +### Know what throws + +`fetchGetTrip` and `executeGetTrip` throw on a failed request. The readable +error an agent sees is produced one level up, in the generated registration +wrapper, which catches the throw and returns `asToolError(error)`. + +So a unit test of the raw caller should assert the throw: + +```ts +it("throws on a failed response", async () => { + vi.stubGlobal( + "fetch", + vi.fn(async () => new Response("nope", { status: 500, statusText: "Server Error" })), + ); + + await expect(fetchGetTrip({ id: "trip_1" })).rejects.toThrow("500"); +}); +``` + + + +### Test a journey's logic + +The draft and the gate belong to `journey.webmcp.ts`, so leave them alone. +Export the two functions you wrote and test those on their own. + +```ts title="src/webmcp/journeys/book-ride.webmcp.ts" +import { createJourney } from "../journey.webmcp"; +import { geocodeAddressTool, fetchGeocodeAddress } from "../geocode-address.webmcp"; +import { executeRequestRide, type RequestRideInput } from "../request-ride.webmcp"; + +export const storeDestination = (place: unknown) => ({ destination: place }); +export const buildRideInput = (draft: Readonly>) => + draft as RequestRideInput; + +export const bookRide = createJourney({ + name: "book-ride", + goal: "Book a ride to a destination the user gives", + steps: { + "resolve-destination": { + tool: geocodeAddressTool, + call: (input, signal) => fetchGeocodeAddress(input as never, signal), + store: storeDestination, + provides: ["destination"], + }, + // ... + }, + submit: { + description: "Request the ride and show the driver's details.", + build: buildRideInput, + run: executeRequestRide, + }, +}); +``` + +Now the parts with logic are ordinary functions: + +```ts title="src/webmcp/journeys/book-ride.test.ts" +import { expect, it } from "vitest"; +import { buildRideInput, storeDestination } from "./book-ride.webmcp"; + +it("stores the resolved place under its draft field", () => { + const place = { lat: 37.6, lng: -122.4, label: "SFO" }; + expect(storeDestination(place)).toEqual({ destination: place }); +}); + +it("assembles the ride request from a full draft", () => { + const input = buildRideInput({ + destination: { lat: 37.6, lng: -122.4 }, + pickupAt: "2026-03-14T19:30:00Z", + }); + expect(input).toMatchObject({ pickupAt: "2026-03-14T19:30:00Z" }); +}); +``` + + + +### Test the whole thing in a browser + +The one thing unit tests cannot cover is the real thing: registration, the +browser validating input against the schema, the confirmation dialog, and the +page updating. That needs a browser with the signed-in session. + +Open your app with WebMCP enabled, use the DevTools panel to call the tools by +hand, and go all the way through the submit gate, including the confirmation. +See [Chrome DevTools WebMCP panel](/docs/devtools). + + + + +## What not to test + +- **The generated region.** It is derived from your spec and regenerated. Test + the spec indirectly through `verify`, and test your `execute` bodies. +- **The runtime helpers.** `callApi`, `toolResult`, and `asToolError` are + library code. If you find a bug in them, fix it in your repo and send it + upstream; do not snapshot their output. +- **The draft's lifetime.** That the draft resets on reload is a documented + behavior, not something to assert in a unit test. + +## Put the mechanical checks in CI + +Unit tests cover logic. `verify` covers the surface itself: names, description +budgets, the journey gate. Run both in CI. + +```bash +npx @webmcp-stack/codegen verify +``` + +## Next + + + + Where testing fits in the wider loop. + + + What to check on screen when a tool runs. + + + The common first-run failures. + + diff --git a/site/content/docs/tool-or-journey.mdx b/site/content/docs/tool-or-journey.mdx new file mode 100644 index 0000000..d666cfe --- /dev/null +++ b/site/content/docs/tool-or-journey.mdx @@ -0,0 +1,105 @@ +--- +title: Tool, group, or journey? +description: Three shapes for three different problems. Pick the wrong one and the agent gets more surface than it can use. +--- + +The surface an agent uses well is small and intent-shaped. When you add a +capability, it takes one of three shapes. This page is how to tell which. + +## The short answer + +| The problem | The shape | What it looks like | +| --- | --- | --- | +| One action, one call | **A tool** | `cancel-order`, `search-products` | +| One action the API split into two calls | **A grouped tool** | `upload-media` wrapping request-upload and complete-upload | +| A goal that takes several calls, shares state, and ends in one write | **A journey** | `book-ride`: resolve, schedule, confirm | + +## A tool + +Most things are a tool. If an agent can express the whole action as one call +with the inputs it already has, you are done. The generator makes these from +your spec, and reads work immediately. + +The question to ask is not "does this endpoint exist" but "would a helpful +person do this in one step". If yes, it is a tool. + +## Grouping + +Some endpoints only make sense together. Requesting an upload and completing it +are one action that the API happens to split in two calls. An agent should never +have to orchestrate that handshake, so the generator detects the pair and emits +one coarse, withheld tool next to the members. + +You do not usually decide this. Grouping happens at generate time from the spec, +merges only when the rules are exact, and skips anything fuzzy with a note. You +adopt the proposal by enabling it. + +## A journey + +Reach for a journey when all of these are true: + +- **The goal takes more than one call.** Resolve a destination, then set a time, + then request the ride. +- **The steps share state.** A later step or the submit needs what an earlier + step produced. +- **At least one input cannot be invented.** The ride needs coordinates, which + only the geocoder can provide. An agent that guesses gets rejected. +- **The end is a real write that a person should confirm.** + +If a goal has the first two but not the last, you may still want a journey: it +is how you keep a multi-call flow from becoming several loose tools the agent +has to sequence itself. + +The [journeys page](/docs/journeys) is the walkthrough. + +## Signs it should not be a journey + +- **One call.** That is a tool. A one-step journey is a tool with extra steps. +- **No shared state.** If each step is independent, the agent can just call the + tools. Wrapping them adds surface without adding safety. +- **More than five steps.** Past five, you are usually describing two journeys. +- **It is really about navigation.** Moving the page around is app behavior, not + a tool call. Keep it in your UI. + +## Signs it should not be a tool + +- **The name mirrors the route.** `post-v1-trips-id-cancel` is an endpoint + wearing a name. The intent is `cancel-trip`. +- **It only makes sense inside a flow.** If calling it alone leaves the app in a + broken state, it belongs to a journey, not the public surface. +- **Two tools are really one decision.** If an agent always calls them together, + they are one tool. + + + + The draft is one object per page load, so two parallel attempts at the same + journey share it and overwrite each other. That is accepted, not handled. + Agents work one thing at a time in practice, and a reload starts clean. + + + Read it from that journey's draft is not supported, because drafts are + private to a journey. If two flows genuinely share state, they are probably + one journey. If they do not, pass the value from the first submit's result + into the second journey's step as normal input. + + + Whether a step navigates is app behavior, not a tool concern. The journey's + submit runs your real tool, so whatever that tool does to the page when a + person clicks the button is what should happen here too. See + [Make the effect visible](/docs/visible-effects). + + + +## Next + + + + The multi-step primitive, with a full example. + + + How classification and withholding decide what agents can reach. + + + The loop that turns the surface into one you trust. + + diff --git a/site/content/docs/troubleshooting.mdx b/site/content/docs/troubleshooting.mdx index a5109d0..b8c7e58 100644 --- a/site/content/docs/troubleshooting.mdx +++ b/site/content/docs/troubleshooting.mdx @@ -1,6 +1,7 @@ --- title: Troubleshooting description: Common errors and how to fix them. +--- ## "My tools 404 in DevTools" diff --git a/site/content/docs/visible-effects.mdx b/site/content/docs/visible-effects.mdx index e4e3e9b..ffb8795 100644 --- a/site/content/docs/visible-effects.mdx +++ b/site/content/docs/visible-effects.mdx @@ -3,14 +3,28 @@ title: Make the effect visible description: An agent acts while a human watches the page. A call that changes something should change it on screen, not just in the response. --- -# Make the effect visible - When an agent calls one of your tools, a person is usually watching the page. If the call changes something and the screen does not move, the person sees nothing happen, and the two experiences of your app drift apart: the agent's and the human's. So a tool's job is not finished when the request succeeds. It is finished when the page -reflects it. +reflects it. One step in the call is easy to forget, because nothing fails when you skip it: + +```mermaid +sequenceDiagram + actor User as Human + actor Agent as Agent + participant Page as Page (WebMCP) + participant API as Your API + + Agent->>Page: call a tool + Note over Page: the browser validates the input against the schema + Page->>API: execute() calls your real endpoint + API-->>Page: the response + Note over Page: update the UI: navigate, invalidate a query, dispatch an event + Page-->>Agent: toolResult + Note over User: the human sees the change +``` ## The pattern diff --git a/site/content/docs/why-agent-tools.mdx b/site/content/docs/why-agent-tools.mdx new file mode 100644 index 0000000..c830061 --- /dev/null +++ b/site/content/docs/why-agent-tools.mdx @@ -0,0 +1,118 @@ +--- +title: Why make your site agent-ready +description: Agents already read your pages and guess. A small set of well-described tools is how your site takes part on purpose, instead of by accident. +--- + +## Agents are already using your site + +An agent is a program that reads a page and then acts. It books the flight, files +the expense, cancels the subscription, checks the order. People are already +pointing agents at the sites they use, and those agents are reading your HTML +and guessing which button means "refund". + +That guess is the problem. HTML was built to be rendered for a person, not +called by a program. The agent sees a wall of markup, infers that the big red +button is the destructive one, and sometimes it is right. WebMCP is the +browser's answer: a page can register a small set of typed tools, with names, +descriptions, and input schemas, that an agent calls directly, in the signed-in +user's session. + +So the choice is not whether agents will act on your site. It is whether they +act through something you described, or through something they inferred. + +## A few good tools beat a mirror of your API + +The instinct is to expose everything. Resist it. + +A tool's description is part of the prompt the model reasons over. Fifty tools +make the model choose worse, not better. The surface agents use well is small +and intent-shaped: + +- `search-flights` instead of `get-v1-flights` +- `cancel-order` instead of `delete-order-item` +- `document-trip` instead of `post-trips` plus `post-trips-id-media` + +Think about the handful of things a helpful person would do on your site for +someone else. Those are the tools. The rest is API you keep for yourself. + +## It is a safety decision, not just a feature + +The caller is a model acting as your user, and it may be reading page content an +attacker influenced. Every tool you expose is both an API and a security +surface, and more tools means more chances to pick the wrong one. A few rules +follow from that, and this tool applies them by default: + +- Reads can work immediately. They change nothing. +- Writes stay off until a human turns them on, one at a time. +- Destructive actions ask the user to confirm before every call. +- The schema is not the security boundary. Your server still validates every + call, because the tool calls your real endpoint, not a generated shortcut. + +## What this tool does for you + +`@webmcp-stack/codegen` turns the contract you already have, an OpenAPI spec or +your validation schemas, into real TypeScript files in your repo: + +- It names and describes each endpoint the way an agent reads it, and writes + the input schema from the types you already defined. +- It classifies every endpoint read, write, or destructive, and withholds the + ones an agent should not see yet. +- It generates the user-confirmation step for mutations in a region you cannot + accidentally edit away. +- It scaffolds a skill file so your own coding agent writes the pieces that need + product knowledge, like journeys, in the shape this tool expects. +- It audits the result, in plain language, and `verify` fails CI when something + is wrong. + +## Who this is for + +This tool generates from a contract you maintain: an OpenAPI spec, or the +validation schemas your app already uses. If you have one, it turns that into a +safe agent surface. If you do not, writing it is the work to do first. + +That is deliberate. It does not read your code and guess at intent, because a +tool is only as good as the source behind it: a vague or stale contract makes +vague or stale tools. A generator that claims to work on any codebase, by +inferring meaning from source, tends to work well nowhere. We would rather do +this one thing well. + +## What it does not do + +Being clear about the edges is part of using it well: + +- **It does not decide your product's intents.** An OpenAPI spec says + `POST /v1/trips` exists. It does not say that creating a trip is really + "search, set details, confirm". That lives in your product, and a person or + their agent has to write it. See [Working with your coding + agent](/docs/working-with-your-agent). +- **It does not make an unsafe API safe.** It calls your real endpoint on + purpose, so your validation, permissions, and rate limits still do the real + work. +- **It does not guarantee an agent uses your tools well.** Good names and + descriptions make it far more likely. Nothing makes it certain. +- **It does not run a model.** Generation is deterministic. No key, no network, + no surprises between two runs. + +## Honesty about where WebMCP is + +WebMCP is early, and the specification is still moving. It may change in ways +that affect what you generated: this project pins the draft it targets and +reports spec drift, but treat the generated surface as something you review, +not something you set and forget. + +That is also why the output is plain files in your repo with no runtime +dependency. If this tool disappears tomorrow, your tools keep working. + +## Where to go next + + + + Generate a working surface in about five minutes. + + + The tools exist. Now decide what agents may actually do, and keep it honest. + + + How to prompt an agent to extend the surface and write journeys. + + diff --git a/site/content/docs/working-with-your-agent.mdx b/site/content/docs/working-with-your-agent.mdx new file mode 100644 index 0000000..c4b4395 --- /dev/null +++ b/site/content/docs/working-with-your-agent.mdx @@ -0,0 +1,97 @@ +--- +title: Working with your coding agent +description: The generator writes the parts an API can describe. Your own agent writes the parts that need product knowledge, including journeys. Here is how to ask it. +--- + +## The generator and your agent do different jobs + +The generator is good at what the API contract already states: the method, the +path, the types, the risk, the confirmation. It cannot know that "create a trip" +on your site means *search for the place, set the dates, then confirm*, or that +your product is a journal for trips people took rather than a planner for trips +they might take. + +That knowledge lives with you, and your coding agent can help you write it down, +if you give it the context the tool cannot. + +## The skill file + +Every `generate` run writes a skill file to your repo: + +```text +.agents/skills/webmcp-tools/SKILL.md +``` + +It is the rules this tool expects, written for an agent to read: the naming +rules, the character budgets, the description format, the exposure decisions, +the execute contract, and the journey pattern. It is regenerated on every run, +so it stays in step with the version you have. + +Agent tools discover it at that path: AGENTS.md, Claude Code, and the generic +"skills" standard all look there. If your project has its own skill directory, +skills stack, so your project rules sit on top of these. + +You do not have to do anything to activate it. A capable coding agent opens the +repo, and the rules are in front of it. Your job is to ask for the right thing. + +There is a page of ready-to-use prompts for the recurring jobs in the +[Prompt cookbook](/docs/prompt-cookbook). The rest of this page explains why +they work and how to check the result. + +## What to give the agent + +Three things turn a vague request into a good tool or journey: + +1. **The product context.** What the site is, who uses it, and when in their +life the action happens. This is the part the generator cannot see, and it is +where wrong-but-plausible names come from. +2. **A pointer to the skill file**, so the agent follows the naming, budget, and +safety rules instead of inventing its own. +3. **The existing generated tools**, so the agent composes them instead of +re-implementing the API. + +## What to check when the agent is done + +An agent can be confidently wrong, especially about intent. Before you accept +its work: + +- **Read the diff.** Did it edit inside a generated region? If so, ask it to + move the change to `.webmcp-codegen.json` instead, or it will be overwritten. +- **Run `verify`.** It catches the mechanical mistakes: a description over + budget, a journey with no submit gate, a direct `fetch` that bypasses the + generated callers. +- **Check the safety, not just the syntax.** A step tool should be a read. If a + step calls something that writes, the write is not behind the submit gate, and + the human confirmation is not in the path. Ask the agent to move it into the + submit. +- **Check the name against your product.** A name can pass every mechanical + check and still be wrong for what the flow actually does. This is the one no + linter catches, and it is why the skill file keeps asking the agent to + understand the product first. + + + A journey is only as safe as the code in it. The submit gate, the + confirmation, and the draft clearing live in the generator-owned + `journey.webmcp.ts` and cannot be edited away. A step's `call` and `run`, on + the other hand, are ordinary code. Keep writes in the submit. + + +## Do not let the agent decide exposure + +The agent can propose that a write be enabled. It should not do it on its own. +Enabling a write is a human decision about what a model may do on your site +while acting as your user. Keep that click. + +## Next + + + + The multi-step primitive, end to end. + + + The full loop, from reviewing the surface to watching it in CI. + + + How classification and withholding actually work. + + diff --git a/site/lib/site-url.ts b/site/lib/site-url.ts new file mode 100644 index 0000000..d4d8572 --- /dev/null +++ b/site/lib/site-url.ts @@ -0,0 +1,7 @@ +/** + * The canonical production origin. One place to change it: page metadata, the + * sitemap, robots.txt, and the llms.txt feeds all read from here. + * + * Set NEXT_PUBLIC_SITE_URL to override it (previews, a custom domain move). + */ +export const SITE_URL = process.env.NEXT_PUBLIC_SITE_URL ?? "https://webmcp.souravinsights.com"; diff --git a/site/package.json b/site/package.json index 84eb6e4..1018f52 100644 --- a/site/package.json +++ b/site/package.json @@ -17,7 +17,9 @@ "fumadocs-mdx": "^11.5.0", "fumadocs-ui": "^15.0.0", "lucide-react": "^1.34.0", + "mermaid": "^12.0.0", "next": "^15.3.0", + "next-themes": "^0.4.6", "react": "^19.0.0", "react-dom": "^19.0.0" }, diff --git a/site/source.config.ts b/site/source.config.ts index 2f022f4..32a478c 100644 --- a/site/source.config.ts +++ b/site/source.config.ts @@ -1,3 +1,4 @@ +import { remarkMdxMermaid } from "fumadocs-core/mdx-plugins"; import { defineConfig, defineDocs } from "fumadocs-mdx/config"; export const docs = defineDocs({ @@ -6,7 +7,8 @@ export const docs = defineDocs({ export default defineConfig({ mdxOptions: { - remarkPlugins: [], + // ```mermaid blocks become ; see components/mermaid.tsx. + remarkPlugins: [remarkMdxMermaid], rehypePlugins: [], }, });