From bb02da04c3ce03741628426960158d71001664de Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Sun, 1 Nov 2026 22:31:57 +0200 Subject: [PATCH] fix(h5p-author): the skill writes every content type from its type file (#102) The quiz format's two references and its examples leave the skill: a quiz is written from the H5P.QuestionSet and H5P.MultiChoice type files, which cover what the format did. The examples are now fixtures in packages/author/tests/fixtures, since the package still builds the format for sources written for the first skill. AGENTS.md records this, and the single verifier run of b02b13a. Closes #102 Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 28 +++--- apps/skills/tests/author-playback.test.ts | 7 +- apps/skills/tests/author-verify.test.ts | 14 +-- packages/author/tests/build.test.ts | 7 +- .../tests/fixtures}/phishing-scenarios.json | 0 .../author/tests/fixtures}/quiz.json | 0 skills/h5p-author/SKILL.md | 2 - skills/h5p-author/references/authoring.md | 3 - skills/h5p-author/references/quiz-format.md | 65 -------------- skills/h5p-author/references/scenario-quiz.md | 87 ------------------- 10 files changed, 35 insertions(+), 178 deletions(-) rename {skills/h5p-author/assets => packages/author/tests/fixtures}/phishing-scenarios.json (100%) rename {skills/h5p-author/assets => packages/author/tests/fixtures}/quiz.json (100%) delete mode 100644 skills/h5p-author/references/quiz-format.md delete mode 100644 skills/h5p-author/references/scenario-quiz.md diff --git a/AGENTS.md b/AGENTS.md index 0d6a1727..dcec3dd4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -180,13 +180,15 @@ packages/author/ @missing-elements/h5p-author — a playable .h5p of an Scenario's paths, Game Map's stages, Find the Hotspot's areas lib/text.mjs, translate.mjs, media.mjs, pack.mjs, schema.mjs, quiz.mjs text as H5P stores it; the libraries' translations; files and web addresses; the pinned pack read - by part; the field list a source is written from; the skill's quiz format + by part; the field list a source is written from; the first skill's + quiz format, still built scripts/ sync-skill.mjs writes the skill's type files' generated sections from the pack (`pnpm --filter @missing-elements/h5p-author sync:skill`); check-pack-published.mjs, the publish guard for the pinned pack tests/ node tests: the generator, text and translations, layouts and schema, - builds with media from disk and a local server, the command; - package-types.ts as for the normalizer + builds with media from disk and a local server, the command, the quiz + format; fixtures/ the quiz format's two examples, which apps/skills + plays too; package-types.ts as for the normalizer types/ generated, not in git: the declarations packages/cmi5/ @missing-elements/h5p-cmi5 — cmi5 for the element, published as a library @@ -310,9 +312,10 @@ apps/skills/ the skills' test project, a project of the kind a skil tests/author-verify.test.ts every h5p-author type file's example built, with sample media, and played trusted and untrusted through the verifier's published API, each a pass with no warnings; each file's generated section compared with - the pack's, so a pack refresh shows a change; and the quiz format: both - kinds, the eight-scenario example, a quiz in Ukrainian and one whose - titles and author carry quotes and markup + the pack's, so a pack refresh shows a change; and the quiz format, from + the author package's fixtures: both kinds, the eight-scenario example, + a quiz in Ukrainian and one whose titles and author carry quotes and + markup tests/author-playback.test.ts the eight-scenario example in the verifier's preview, the review the skill hands a person, trusted and untrusted: every question's feedback, 75% failing and 83.3% passing an 80% threshold, retry to 100%, and only @@ -1958,10 +1961,15 @@ owner's design: the content type chosen with the user first, a file per type in skill so no agent works out how a type is built, a short skill, one package for the whole pack. It replaced the skill's bundled `scripts/build-quiz.mjs`, which built Multiple Choice and Question Set alone, and its quiz format stays a source the build takes (`lib/quiz.mjs` maps it), so nothing -written in it broke. Measured on a five-question quiz, at four characters a token: `content.json` -was about 3,470 tokens to write by hand, after reading 3,900 to 9,500 of a type's semantics; the -source is 1,000 to 1,800 tokens, written from a type file of 200 to 3,000. Seven things the code -does not say by itself: +written in it broke. The skill stopped describing it on 2026-11-01 (#102), when an agent given a +quiz was found reading both it and the two type files: every type is written from its type file, +a quiz from H5P.QuestionSet's and H5P.MultiChoice's, and the format's two examples are the tests' +fixtures. The skill runs the verifier once, trusted (the owner's call, the same day): the build +already holds text to the content check's rules, from the pack an untrusted check plays, and +`apps/skills` plays every example untrusted too. Measured on a five-question quiz, at four +characters a token: `content.json` was about 3,470 tokens to write by hand, after reading 3,900 to +9,500 of a type's semantics; the source is 1,000 to 1,800 tokens, written from a type file of 200 +to 3,000. Seven things the code does not say by itself: - **One generator, two ways to run it.** `ContentGenerator` was the sweep's (`apps/demo/scripts/lib/sweep-content.mjs`), and moved here so there is one copy; the sweep runs diff --git a/apps/skills/tests/author-playback.test.ts b/apps/skills/tests/author-playback.test.ts index b0cd23bf..145b3185 100644 --- a/apps/skills/tests/author-playback.test.ts +++ b/apps/skills/tests/author-playback.test.ts @@ -9,13 +9,14 @@ import { buildPackage } from '@missing-elements/h5p-author' const project = resolve(import.meta.dirname, '..') // The pack this project installed, named to the preview as `--libraries` names a copy of its parts. const parts = resolve(project, 'node_modules/@missing-elements/h5p-libraries/parts') -/** The skill's quiz format, as the scenario example writes it. */ +/** The quiz format the skill first wrote, as the scenario example writes it. */ type Quiz = { kind: string, passPercentage?: number, passedFeedback?: string, failedFeedback?: string, questions: { title?: string, prompt: string, multipleAnswers?: boolean, answers: { text: string, correct: boolean, feedback?: string }[] }[] } -const source: Quiz = JSON.parse(await readFile(resolve(project, '../../skills/h5p-author/assets/phishing-scenarios.json'), 'utf8')) -const example: Quiz = JSON.parse(await readFile(resolve(project, '../../skills/h5p-author/assets/quiz.json'), 'utf8')) +const fixtures = resolve(project, '../../packages/author/tests/fixtures') +const source: Quiz = JSON.parse(await readFile(join(fixtures, 'phishing-scenarios.json'), 'utf8')) +const example: Quiz = JSON.parse(await readFile(join(fixtures, 'quiz.json'), 'utf8')) const single: Quiz = { ...example, kind: 'multiple-choice', questions: [example.questions[0]] } let dir: string let file: string diff --git a/apps/skills/tests/author-verify.test.ts b/apps/skills/tests/author-verify.test.ts index 034b9db1..41db3490 100644 --- a/apps/skills/tests/author-verify.test.ts +++ b/apps/skills/tests/author-verify.test.ts @@ -10,10 +10,11 @@ import { afterAll, beforeAll, describe, expect, it } from 'vitest' const project = resolve(import.meta.dirname, '..') // The pack this project installed, named to the check as `--libraries` names a copy of its parts. const parts = resolve(project, 'node_modules/@missing-elements/h5p-libraries/parts') -const skill = resolve(project, '../../skills/h5p-author') -const types = join(skill, 'references/types') -const example = JSON.parse(await readFile(join(skill, 'assets/quiz.json'), 'utf8')) -const scenarios = JSON.parse(await readFile(join(skill, 'assets/phishing-scenarios.json'), 'utf8')) +const types = resolve(project, '../../skills/h5p-author/references/types') +// The quiz format's two examples, which the author package's tests keep. +const fixtures = resolve(project, '../../packages/author/tests/fixtures') +const example = JSON.parse(await readFile(join(fixtures, 'quiz.json'), 'utf8')) +const scenarios = JSON.parse(await readFile(join(fixtures, 'phishing-scenarios.json'), 'utf8')) const META = { title: 'Example', language: 'en', license: 'CC BY', authors: [{ name: 'Ann Author', role: 'Author' }] } const files = (await readdir(types)).filter((name) => name.endsWith('.md')).sort() let dir: string @@ -88,7 +89,10 @@ beforeAll(async () => { }) afterAll(async () => { if (dir) await rm(dir, { recursive: true, force: true }) }) -/** Step 4 of the skill: the check, through the verifier's published API, trusted and untrusted. */ +/** + * Step 4 of the skill, the check, through the verifier's published API: trusted, as the skill + * runs it, and untrusted, which holds the build to the content check's rules. + */ async function playsBothWays(file: string) { for (const untrusted of [false, true]) { const report = await verifyPackage({ file, untrusted, libraries: untrusted ? parts : undefined, settle: 500 }) diff --git a/packages/author/tests/build.test.ts b/packages/author/tests/build.test.ts index e9ad257b..c2882b35 100644 --- a/packages/author/tests/build.test.ts +++ b/packages/author/tests/build.test.ts @@ -14,11 +14,12 @@ import { MediaStore } from '../lib/media.mjs' import { quizProblems } from '../lib/quiz.mjs' import { jpeg, mp4, png } from './helpers/media' -const skill = resolve(import.meta.dirname, '../../../skills/h5p-author') +// The quiz format's two examples, from the skill that first wrote it. +const fixtures = resolve(import.meta.dirname, 'fixtures') const parts = resolve(import.meta.dirname, '../../libraries/parts') const bin = resolve(import.meta.dirname, '../bin/h5p-author.mjs') -const example = JSON.parse(await readFile(join(skill, 'assets/quiz.json'), 'utf8')) -const scenarios = JSON.parse(await readFile(join(skill, 'assets/phishing-scenarios.json'), 'utf8')) +const example = JSON.parse(await readFile(join(fixtures, 'quiz.json'), 'utf8')) +const scenarios = JSON.parse(await readFile(join(fixtures, 'phishing-scenarios.json'), 'utf8')) const exec = promisify(execFile) const META = { title: 'Rivers', language: 'en', license: 'CC BY', authors: [{ name: 'Ann Author', role: 'Author' }] } let dir: string diff --git a/skills/h5p-author/assets/phishing-scenarios.json b/packages/author/tests/fixtures/phishing-scenarios.json similarity index 100% rename from skills/h5p-author/assets/phishing-scenarios.json rename to packages/author/tests/fixtures/phishing-scenarios.json diff --git a/skills/h5p-author/assets/quiz.json b/packages/author/tests/fixtures/quiz.json similarity index 100% rename from skills/h5p-author/assets/quiz.json rename to packages/author/tests/fixtures/quiz.json diff --git a/skills/h5p-author/SKILL.md b/skills/h5p-author/SKILL.md index 0e04c7ab..55794076 100644 --- a/skills/h5p-author/SKILL.md +++ b/skills/h5p-author/SKILL.md @@ -70,8 +70,6 @@ content holds have files too: H5P.AdvancedText, H5P.Image, H5P.Video, H5P.Text. - A file is `{ "path": "photo.jpg" }`, beside the source or a web address, with `copyright` when known. - Sub-content is `{ "library": "H5P.MultiChoice", "params": { … }, "metadata": { "title": "…" } }`. -- A quiz of Multiple Choice questions alone is shorter in [the quiz format](./references/quiz-format.md), - which the build also takes. ## 3. Build diff --git a/skills/h5p-author/references/authoring.md b/skills/h5p-author/references/authoring.md index 59ebe24f..5d44e804 100644 --- a/skills/h5p-author/references/authoring.md +++ b/skills/h5p-author/references/authoring.md @@ -31,9 +31,6 @@ Use what the user already gave. Ask for what is missing and matters: - A pass mark with its messages: the failed one says what to do next. - Alternative text for every picture that carries meaning: what it shows, for a learner who cannot see it. -- For a quiz with a pass mark and mixed answer modes, [the eight-scenario brief](./scenario-quiz.md) - and [its source](../assets/phishing-scenarios.json) show the whole of it: answer key, - explanatory feedback, an 80% threshold, and the fail, retry and pass checks. ## Media diff --git a/skills/h5p-author/references/quiz-format.md b/skills/h5p-author/references/quiz-format.md deleted file mode 100644 index ebd6db94..00000000 --- a/skills/h5p-author/references/quiz-format.md +++ /dev/null @@ -1,65 +0,0 @@ -# The quiz format - -A shorter source for a Multiple Choice question, or a Question Set of them alone, which the -build takes in place of `{ "type", "params" }`: it was the skill's only format until 2026-11, and -stays. For any other question in a set, write the set's `params` from -[its type file](./types/H5P.QuestionSet.md). Unknown fields and wrong types fail with a path; -nothing is silently dropped. All text is plain text: `