diff --git a/design/author.md b/design/author.md index 40e9e20b..6d01c7a6 100644 --- a/design/author.md +++ b/design/author.md @@ -15,7 +15,7 @@ already holds text to the content check's rules, from the pack an untrusted chec `apps/skills` plays every example untrusted too. Measured on a five-question quiz, at four characters a token: `content.json` was about 3,470 tokens to write by hand, after reading 3,900 to 9,500 of a type's semantics; the source is 1,000 to 1,800 tokens, written from a type file of 200 -to 3,000. Fourteen things the code does not say by itself: +to 3,000. Eighteen things the code does not say by itself: - **One generator, three ways to run it.** The third, `saved`, checks content as H5P stores it (*Saved content is checked as the build checks what it writes*, below). `ContentGenerator` was the sweep's @@ -319,6 +319,84 @@ to 3,000. Fourteen things the code does not say by itself: where the verifier's preview opens: a real browser, in which the player runs as it is, video included. The owner chose the panel, with media to come, and the card was not built. +- **A problem says what to write, and what is in the wrong place is checked all the same** + (2026-11-10, #172). The owner gave the skill to Gemini in Chrome, which could not build a simple + Interactive Book and quoted `params.chapters: has 0 items; it takes at least 1`. A chat there has + the skill's text, no type file and nothing to run, so it learns a type's shape from the build's + problems alone. Measured that way with the weakest Claude model, Haiku 5.5: SKILL.md's text, each + reply built as the build page builds it, its problems pasted back in the page's words, up to four + rounds; an Interactive Book, a Course Presentation, a Column, a Question Set and a Branching + Scenario, three runs each, the same 15 first replies under each set of messages. With 0.6.0's + messages 3 built at once, the Question Sets, 8 by the second round, 12 by the third, 13 by the + fourth, and two scenarios never. What cost the rounds was a type's levels, not its fields: + - Sub-content in the place of the group that holds it, `{ "library", "params" }` where a + Column's item is `{ "content": { … } }`, a slide's element `{ "action": { … } }` and a + scenario's node `{ "type": { … } }`: 9 of the 24 failed rounds, three or four problems an item, + one of them a list of 35 libraries, none saying what to write. + - A field a level above its group, `params.slides` for `params.presentation.slides`: the build + said it was no field there and checked nothing of what was written in it, so the next round + found what was wrong inside, an `