Skip to content

Enforce the component contract, and stop two other tables lying about their types - #6

Merged
houko merged 1 commit into
mainfrom
fix/enforce-contracts
Sep 26, 2026
Merged

houko merged 1 commit into
mainfrom
fix/enforce-contracts

Conversation

@houko

@houko houko commented Sep 21, 2026

Copy link
Copy Markdown
Member

Three leftovers from the audit, and they were one finding seen three times: a value typed at the boundary and trusted afterwards, with nothing between the model's answer and the code that acts on it.

The catalog declared a contract nobody enforced

lib/catalog.ts opens by saying that nothing outside it can ever appear in a generated page, "that guarantee is the point of the whole design". The guarantee held for component names and not for the props under them. Measured:

catalog.validate({ … Pricing with tiers: "免费" … }).success   // true
catalog.data.components.Pricing.props.safeParse({ …tiers: "免费" })
//  → false, "Invalid input: expected array, received string"

The schema knew. Nothing asked it. So the zod definitions were a description of what the LLM ought to send, and the only thing standing between a wrong-typed field and (props.tiers as {…}[]).map — a cast, not a check — was isReady, which asks whether a value is present and non-blank, not whether it is the right kind of thing. "免费" is present and not blank.

validateProps reads those same schemas, so the contract is enforced by the file that states it and there is no second description of any block's shape to drift out of step. buildSpec consults it next to isReady, and a block that fails holds its place as a skeleton — the same thing the page already does for copy that has not arrived, which is the honest description of copy that came back unusable. Better than the two alternatives: rendering a block that is wrong, or failing a page over one block.

The two client-side splices in applyEdit get the same guard — a restyle and an add assemble their element locally and never pass through compose.

Lookup is Object.hasOwn, not indexing: the component name can arrive from a model, and constructor is not a component.

Fixing that exposed a smaller thing doing real damage

elementsFor coerced a non-array tiers to [] so its eager slice(0, 1).map(...) could not throw. That was itself an audit fix, and it stops the crash — but it also defeats the check, because an empty array is a legal tiers. The block satisfied its schema, rendered, and drew a price section with a heading, a button and no prices.

The guard now covers the operation and not the payload. The value the model actually sent reaches the layer that knows what a tier list looks like, and the crash it was written for still cannot happen.

A rule whose job is counting was counting the wrong thing

layoutRules read .length off fields it declared as unknown[] and JSON.parse filled with whatever the model said:

tiers: "免费"   →   "免费".length === 2   →   pricing_multi

A business with no tiers at all got the three-column comparison layout. It counts with Array.isArray now, and the parameter type says unknown so the lie cannot be written again.

A fallback that was the only thing that noticed

themeVars answers an unrecognised key with forest and says nothing. That is correct for a total function and wrong as the only place anyone finds out: the page rendered green while the decision log, the theme picker and the exported tokens all named something else.

isThemeKey narrows the Choice answer where it enters compose.ts. The fallback stays exactly as it was — the MCP theme_tokens tool deliberately wants it and labels it, verified unchanged:

mcp theme_tokens {"theme":"nope"} → requested: nope, theme: forest, matched: false

One mutation stayed green, and that is in here too

Laundering tiers back to an empty array in pricing_multi failed no test. A wrong-typed tiers stops layoutRules firing, the unpicked-slot fallback then chooses pricing_single, and the multi path was never exercised — so the test I had written only covered half of what it claimed.

lib/content.test.ts now drives both variants by name off SLOTS rather than by hand. Re-run, both directions go red.

mutation result
buildSpec drops the props check 2 red
theme narrowing removed 1 red
layoutRules back to .length 1 red
validateProps indexes instead of Object.hasOwn 1 red
isThemeKey accepts any string 2 red
pricing_multi launders tiers to [] green at first → 1 red after the test was fixed
pricing_single launders tiers to [] 2 red

Verification

npx tsc --noEmit     exit 0
npx vitest run       22 files, 362 tests, all passing   (was 20 files, 347)
npm run build        compiled successfully
npm run mcp          theme_tokens fallback unchanged and still labelled

End to end through the real editor with Playwright: 10 blocks rendered, 0 skeletons left, zero page errors, then a restyle applied on top. The enforcement does not reject valid copy — which is the failure mode a check like this actually has.

15 new tests across lib/catalog.test.ts (new), lib/content.test.ts (new), lib/compose.test.ts, lib/plan.test.ts and lib/themes.test.ts.

One test changed meaning, deliberately

lib/compose.test.ts asserted Array.isArray(props.tiers) === true for the wrong-typed case. That pinned the coercion rather than the outcome — it passed precisely because the string had been replaced with []. It now asserts the block is a skeleton, with the reason recorded in the test.

…about their types

Three findings were left over from the audit and they were one finding seen three times: a value typed at the boundary and trusted afterwards, with nothing between the model's answer and the code that acts on it.

`lib/catalog.ts` calls itself the component contract and its header claims nothing outside it can appear in a generated page. That held for component names and not for the props under them. `catalog.validate()` checks the spec's shape and stops there — a `Pricing` element whose `tiers` is the string `"免费"` returns `success: true` from it, measured. The zod schemas were therefore a description of what the LLM ought to send rather than a check on what it did send, and the only thing between a wrong-typed field and `(props.tiers as ...).map` was `isReady`, which asks whether a value is present and non-blank, not whether it is the right kind of thing.

`validateProps` closes that, reading the very schemas the file declares, so the contract is enforced by the file that states it and no second description of any block's shape exists to drift. `buildSpec` consults it alongside `isReady`, and a block that fails holds its place as a skeleton — the same thing the page already does for copy that has not arrived, which is the honest description of copy that came back unusable. The two client-side splices in `applyEdit` get the same guard, since a restyle and an add assemble their element locally and never pass through compose.

The catalog lookup is by `Object.hasOwn`: the component name can arrive from a model, and `constructor` is not a component.

Fixing that exposed a smaller thing doing real damage. `elementsFor` coerced a non-array `tiers` to `[]` so its eager `slice(0, 1).map(...)` could not throw. That stops the crash and defeats the check, because an empty array is a legal `tiers`: the block satisfied its schema, rendered, and drew a price section with a heading, a button and no prices. The guard now covers the operation and not the payload, so the value the model actually sent reaches the layer that knows what a tier list looks like.

`layoutRules` read `.length` off fields it declared as `unknown[]` and `JSON.parse` filled with whatever the model said. `"免费"` is two characters long, so a business with no tiers at all got the three-column comparison layout — a rule whose whole job is counting, silently counting the wrong thing. It counts with `Array.isArray` now, and the parameter says `unknown` so nothing else can be written.

`themeVars` answers an unrecognised key with forest and says nothing. That is right for a total function and wrong as the only place anyone finds out: the page rendered green while the decision log, the theme picker and the exported tokens all named something else. `isThemeKey` narrows the Choice answer where it enters, so the fallback stays available for the MCP tool that wants it and labels it, and becomes unreachable in the app.

One mutation stayed green on the first pass and is worth recording: laundering `tiers` back to an empty array in `pricing_multi` failed no test, because a wrong-typed `tiers` stops `layoutRules` firing and the unpicked-slot fallback then chooses `pricing_single`, so the multi path was never exercised. `lib/content.test.ts` now drives both variants by name off `SLOTS` rather than by hand.
@houko
houko merged commit 86d0797 into main Sep 26, 2026
1 check passed
@houko
houko deleted the fix/enforce-contracts branch September 26, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant