From ca08cc2892d7c73d5475b194c69d474afabbd45f Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 23 Oct 2026 04:03:05 +0300 Subject: [PATCH 1/2] feat: h5p-author, a skill that builds and verifies a text-only quiz An agent asked for an H5P quiz could check a package with h5p-verify but had nothing to build one with, and hand-written packages break in ways the files do not show. - skills/h5p-author: the brief, a validated quiz source, the build, the verifier trusted and untrusted, and a look at the screenshot. Text-only Multiple Choice and Question Set, with the limits stated. - scripts/build-quiz.mjs, portable: loads the library pack and zip.js from the calling project and refuses a zip.js older than 2.24. Copies only the selected libraries' runtime dependencies, each part checked against the pack index's SHA-256. Fills fields from the installed semantics, gives each question a UUID subContentId, and never overwrites its output. - Two examples: a two-question starter, and eight phishing scenarios with an answer key and the fail, pass and retry attempts. - Tests: the helper in Node; the scenarios played through the app, trusted and untrusted; every example through the verifier with no warnings. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 13 + README.md | 13 +- apps/demo/tests/author-playback.test.ts | 104 +++++++ apps/demo/tests/author.test.ts | 209 +++++++++++++ docs/verify.md | 9 + packages/verify/tests/author.test.mjs | 49 +++ skills/h5p-author/LICENSE | 21 ++ skills/h5p-author/SKILL.md | 123 ++++++++ .../h5p-author/assets/phishing-scenarios.json | 222 ++++++++++++++ skills/h5p-author/assets/quiz.json | 48 +++ skills/h5p-author/references/quiz-format.md | 40 +++ skills/h5p-author/references/scenario-quiz.md | 86 ++++++ skills/h5p-author/scripts/build-quiz.mjs | 288 ++++++++++++++++++ 13 files changed, 1223 insertions(+), 2 deletions(-) create mode 100644 apps/demo/tests/author-playback.test.ts create mode 100644 apps/demo/tests/author.test.ts create mode 100644 packages/verify/tests/author.test.mjs create mode 100644 skills/h5p-author/LICENSE create mode 100644 skills/h5p-author/SKILL.md create mode 100644 skills/h5p-author/assets/phishing-scenarios.json create mode 100644 skills/h5p-author/assets/quiz.json create mode 100644 skills/h5p-author/references/quiz-format.md create mode 100644 skills/h5p-author/references/scenario-quiz.md create mode 100644 skills/h5p-author/scripts/build-quiz.mjs diff --git a/AGENTS.md b/AGENTS.md index 47b0d8ce..e0f7b177 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -157,6 +157,8 @@ packages/verify/ @missing-elements/h5p-verify — plays a package headl lib/verify.mjs the static server, the browser run and the report; lib/page.html the page it drives tests/ node tests that run the real thing against the demo's committed content; package-types.ts as for the normalizer + author.test.mjs also builds both quiz kinds with h5p-author and plays + each trusted and untrusted against the local pack, with no warnings types/ generated, not in git: the declarations, `VerifyReport` among them packages/cmi5/ @missing-elements/h5p-cmi5 — cmi5 for the element, published as a library @@ -213,6 +215,13 @@ apps/demo/ the demo app: the dev server for the whole repository, server; engine.ts and harness.ts its helpers. The demo's other pages have no tests (see *Commands*) tests/sweep-content.test.ts the sweep's content generator, in Node + tests/author.test.ts the portable h5p-author helper, its quiz validation, archives and + execution from an installed skill outside the checkout, in Node; + corrupt library parts must fail their SHA-256 before publication, + and a zip.js older than 2.24 must stop it with the install command + tests/author-playback.test.ts the eight-scenario example through the app, trusted and untrusted: + every question's feedback, 75% failing and 83.3% passing an 80% threshold, retry to 100%, + and only the parent Question Set finishing cmi5-catapult/ docker-compose.yml for ADL's CATAPULT player, MySQL and the SQL LRS; scripts/cmi5-catapult.mjs clones CATAPULT into it (ignored) and proves the cmi5 page against the player @@ -262,6 +271,10 @@ skills/ agent skills, one directory per skill, at the root bec `npx skills add ` looks: h5p-verify (when to run the verifier and how to read its report), h5p-player-setup (putting the player on a site), h5p-normalize (when a package needs rewriting to stream, and how to read the dry run) + and h5p-author (text-only Multiple Choice and Question Set from a prompt: + a quiz JSON source, the installed pack's semantics and runtime dependency + closure, then the existing verifier and screenshot review; its bundled + scripts/build-quiz.mjs resolves dependencies from the caller's project) api/no-range.js the Vercel function that stands in for a host without Range on the demo vercel.json the deployment: build command, the /no-range rewrite, caching and security headers diff --git a/README.md b/README.md index 99163337..384d9029 100644 --- a/README.md +++ b/README.md @@ -63,7 +63,7 @@ something else. - **Comes with tools for pipelines.** [`h5p-verify`](https://github.com/missing-elements/h5p-offline-player/blob/main/docs/verify.md) plays a package headless and says whether it works, [`h5p-normalize`](https://github.com/missing-elements/h5p-offline-player/blob/main/docs/streaming-video.md) rewrites one so its video streams, and [`h5p-embed`](https://github.com/missing-elements/h5p-offline-player/tree/main/packages/embed#readme) - writes the embed site. The [agent skills](#agent-skills) cover all three. + writes the embed site. The [agent skills](#agent-skills) cover those workflows and quiz authoring. - **Is tested where it runs.** CI runs the browser tests in Chromium, Firefox and WebKit on every change, and plays every content type in the library pack, untrusted and trusted, on each change to the player, the runtime or the pack, and every week. @@ -286,8 +286,17 @@ is an example. npx skills add missing-elements/h5p-offline-player ``` +[`h5p-author`](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) +turns a prompt or lesson into a text-only Multiple Choice or Question Set: a portable helper +builds the `.h5p` against the installed library pack, and the agent verifies it in the real +player and reviews the screenshot before handing it over. Works with Claude Code and other +agents with filesystem and command access; no provider API key or H5P server is needed. +For example: "Create five practice questions from this lesson for new employees, with +explanatory feedback. Use h5p-author and verify the result." Needs Node 22 or later. + `h5p-player-setup` puts H5P on a site, `h5p-normalize` fixes packages whose video will not -stream, `h5p-verify` checks a generated package plays. +stream, and `h5p-verify` checks a generated package plays. A verifier pass checks startup and +rendering, not whether the answers are right or the activity is accessible. ## Development diff --git a/apps/demo/tests/author-playback.test.ts b/apps/demo/tests/author-playback.test.ts new file mode 100644 index 00000000..a4527a40 --- /dev/null +++ b/apps/demo/tests/author-playback.test.ts @@ -0,0 +1,104 @@ +import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import type { Browser, Page } from 'playwright' +import type { ViteDevServer } from 'vite' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { buildQuiz, type Quiz } from '../../../skills/h5p-author/scripts/build-quiz.mjs' +import { browserType } from './engine' +import { packageHost, playerIs, startSite } from './harness' + +const project = resolve(import.meta.dirname, '..') +const source: Quiz = JSON.parse(await readFile(resolve(project, '../../skills/h5p-author/assets/phishing-scenarios.json'), 'utf8')) +let dir: string +let file: string +let vite: ViteDevServer +let origin: string +let browser: Browser +let elsewhere: Awaited> + +beforeAll(async () => { + dir = await mkdtemp(join(tmpdir(), 'h5p-author-interaction-')) + file = join(dir, 'scenarios.h5p') + await buildQuiz(source, file, project) + ;({ vite, origin } = await startSite()) + browser = await browserType.launch() + elsewhere = await packageHost({ '/scenarios.h5p': file }) +}) +afterAll(async () => { + await browser?.close() + await vite?.close() + await elsewhere?.close() + if (dir) await rm(dir, { recursive: true, force: true }) +}) + +type ResultsWindow = Window & typeof globalThis & { authorFinished: unknown[] } + +async function finished(page: Page, count: number) { + await page.waitForFunction(count => (window as ResultsWindow).authorFinished.length === count, count) + return page.evaluate(() => (window as ResultsWindow).authorFinished.at(-1)) +} + +async function attempt(page: Page, correctQuestions: number, previousResults: number) { + const frame = page.frameLocator('h5p-player iframe') + for (let i = 0; i < source.questions.length; i++) { + const question = source.questions[i] + const candidates = question.answers.filter(answer => i < correctQuestions ? answer.correct : !answer.correct) + const selected = question.multipleAnswers ? candidates : candidates.slice(0, 1) + for (const answer of selected) { + await frame.getByRole(question.multipleAnswers ? 'checkbox' : 'radio').filter({ hasText: answer.text }).click() + } + await frame.getByRole('button', { name: /^Check the answers/ }).click() + for (const answer of selected) { + if (!answer.feedback) throw new Error('The scenario example must explain every answer') + await frame.getByText(answer.feedback, { exact: true }).waitFor({ state: 'visible' }) + } + // Sub-question answers must not finish the parent Question Set. + expect(await page.evaluate(() => (window as ResultsWindow).authorFinished.length)).toBe(previousResults) + await frame.getByRole('button', { name: i === source.questions.length - 1 ? 'Finish' : 'Next', exact: true }).click() + } +} + +it.each([false, true])('plays every scenario across the 80% threshold and retries to 100% (untrusted: %s)', async untrusted => { + const page = await browser.newPage({ viewport: { width: 1100, height: 850 } }) + const errors: string[] = [] + page.on('pageerror', error => errors.push(error.message)) + try { + const path = untrusted ? `/app/?src=${encodeURIComponent(`${elsewhere.origin}/scenarios.h5p`)}` : '/app/' + await page.goto(`${origin}${path}`) + if (!untrusted) await page.locator('input[type=file]').setInputFiles(file) + await playerIs(page, 'ready', 'error') + expect(await page.locator('h5p-player').evaluate(player => player.hasAttribute('untrusted'))).toBe(untrusted) + await page.locator('h5p-player').evaluate(player => { + const results: unknown[] = [] + Object.defineProperty(window, 'authorFinished', { value: results }) + player.addEventListener('finished', event => { + if (event instanceof CustomEvent) results.push(event.detail.statement) + }) + }) + + await attempt(page, 6, 0) + expect(await finished(page, 1)).toMatchObject({ + verb: { id: 'http://adlnet.gov/expapi/verbs/completed' }, + result: { completion: true, success: false, score: { raw: 9, max: 12, scaled: 0.75 } }, + context: { revision: expect.stringMatching(/^sha256:[0-9a-f]{64}/) } + }) + const frame = page.frameLocator('h5p-player iframe') + await frame.getByRole('button', { name: 'Retry', exact: true }).click() + await attempt(page, 7, 1) + expect(await finished(page, 2)).toMatchObject({ + result: { completion: true, success: true, score: { raw: 10, max: 12, scaled: 0.8333 } } + }) + await frame.getByRole('button', { name: 'Retry', exact: true }).click() + await attempt(page, source.questions.length, 2) + expect(await finished(page, 3)).toMatchObject({ + result: { completion: true, success: true, score: { raw: 12, max: 12, scaled: 1 } } + }) + await frame.getByRole('button', { name: 'Show solution', exact: true }).click() + await frame.getByText(source.questions[0].answers[2].text, { exact: true }).waitFor({ state: 'visible' }) + expect(await frame.getByRole('button', { name: /^Check the answers/ }).count()).toBe(0) + expect(errors).toEqual([]) + } finally { + await page.close() + } +}) diff --git a/apps/demo/tests/author.test.ts b/apps/demo/tests/author.test.ts new file mode 100644 index 00000000..effcfe74 --- /dev/null +++ b/apps/demo/tests/author.test.ts @@ -0,0 +1,209 @@ +import { execFile } from 'node:child_process' +import { cp, mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promises' +import { createRequire } from 'node:module' +import { tmpdir } from 'node:os' +import { basename, dirname, join, resolve } from 'node:path' +import { promisify } from 'node:util' +import { BlobReader, TextWriter, ZipReader } from '@zip.js/zip.js' +import { pack } from '@missing-elements/h5p-libraries' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' +import { buildQuiz, validateQuiz } from '../../../skills/h5p-author/scripts/build-quiz.mjs' + +const skill = resolve(import.meta.dirname, '../../../skills/h5p-author') +const project = resolve(import.meta.dirname, '..') +const example = JSON.parse(await readFile(join(skill, 'assets/quiz.json'), 'utf8')) +const scenarios = JSON.parse(await readFile(join(skill, 'assets/phishing-scenarios.json'), 'utf8')) +const exec = promisify(execFile) +const require = createRequire(import.meta.url) +let dir: string + +beforeAll(async () => { dir = await mkdtemp(join(tmpdir(), 'h5p-author-test-')) }) +afterAll(async () => { await rm(dir, { recursive: true, force: true }) }) + +async function archive(path: string) { + const reader = new ZipReader(new BlobReader(new Blob([await readFile(path)])), { checkSignature: true }) + try { + const entries = (await reader.getEntries()).filter(entry => !entry.directory) + const files = new Map() + // Decode every file independently; a copied entry must retain its CRC and compression method. + for (const entry of entries) files.set(entry.filename, await entry.getData(new TextWriter())) + return files + } finally { await reader.close() } +} + +async function copyDependencies(projectDir: string) { + for (const name of ['@missing-elements/h5p-libraries', '@zip.js/zip.js']) { + await cp(dirname(require.resolve(name)), join(projectDir, 'node_modules', name), { + recursive: true, filter: path => basename(path) !== 'node_modules' + }) + } +} + +describe('quiz authoring', () => { + it('accepts both kinds, multiple correct answers, and a zero pass threshold', () => { + expect(() => validateQuiz(example)).not.toThrow() + expect(() => validateQuiz({ ...example, kind: 'multiple-choice', passPercentage: 0, questions: [example.questions[0]] })).not.toThrow() + }) + + it.each([ + [null, 'quiz: expected an object'], + [{ ...example, kind: 'slides' }, 'quiz.kind'], + [{ ...example, title: '' }, 'quiz.title'], + [{ ...example, title: 'a'.repeat(256) }, 'quiz.title'], + [{ ...example, language: 'not_a_tag' }, 'quiz.language'], + [{ ...example, license: 'made-up' }, 'quiz.license'], + [{ ...example, authors: [] }, 'quiz.authors'], + [{ ...example, authors: [{ name: 'A', role: 'Editor' }] }, 'quiz.authors[0].role'], + [{ ...example, passPercentage: 101 }, 'quiz.passPercentage'], + [{ ...example, passPercentage: '50' }, 'quiz.passPercentage'], + [{ ...example, questions: [] }, 'quiz.questions'], + [{ ...example, kind: 'multiple-choice' }, 'exactly one question'], + [{ ...example, customJs: 'evil.js' }, 'quiz.customJs: unsupported field'], + [{ ...example, questions: [null] }, 'quiz.questions[0]'], + [{ ...example, questions: [{ ...example.questions[0], prompt: '' }] }, 'quiz.questions[0].prompt'], + [{ ...example, questions: [{ ...example.questions[0], multipleAnswers: 'yes' }] }, 'quiz.questions[0].multipleAnswers'], + [{ ...example, questions: [{ ...example.questions[0], answers: [] }] }, 'at least two answers'], + [{ ...example, questions: [{ ...example.questions[0], answers: [{ text: 'a', correct: 'true' }, { text: 'b', correct: false }] }] }, '.correct'], + [{ ...example, questions: [{ ...example.questions[0], answers: [{ text: 'a', correct: true }, { text: 'b', correct: true }] }] }, 'exactly one answer'], + [{ ...example, questions: [{ ...example.questions[0], multipleAnswers: true, answers: [{ text: 'a', correct: false }, { text: 'b', correct: false }] }] }, 'at least one answer'], + [{ ...example, questions: [{ ...example.questions[0], answers: [{ text: 'a', correct: true, typo: 'lost' }, { text: 'b', correct: false }] }] }, '.typo: unsupported field'] + ])('rejects malformed input with its field path (%#)', (input, message) => { + expect(() => validateQuiz(input)).toThrow(message) + }) + + it('builds a self-contained Question Set using pack versions, semantics defaults, and the full dependency closure', async () => { + const path = join(dir, 'set.h5p') + const result = await buildQuiz(example, path, project) + const files = await archive(path) + const manifest = JSON.parse(files.get('h5p.json')!) + const content = JSON.parse(files.get('content/content.json')!) + expect([...files.keys()][0]).toBe('h5p.json') + expect([...files.keys()].at(-1)).toBe('content/content.json') + expect(manifest.mainLibrary).toBe('H5P.QuestionSet') + expect(manifest.defaultLanguage).toBe(example.language) + expect(manifest.authors).toEqual(example.authors) + expect(content.passPercentage).toBe(100) + expect(content.texts.finishButton).toBeTruthy() + expect(content.endGame.showResultPage).toBe(true) + expect(content.questions).toHaveLength(2) + expect(content.questions[0].params.question).toContain(example.questions[0].prompt) + expect(content.questions[0].params.UI.checkAnswerButton).toBe('Check') + expect(content.questions[0].params.behaviour.type).toBe('single') + expect(content.questions[1].params.behaviour.type).toBe('multi') + const ids = content.questions.map((question: { subContentId: string }) => question.subContentId) + expect(new Set(ids).size).toBe(2) + expect(ids.every((id: string) => /^[0-9a-f-]{36}$/.test(id))).toBe(true) + expect(result.libraries.length).toBeGreaterThan(2) + expect(result.libraries.length).toBeLessThan(30) + for (const dependency of manifest.preloadedDependencies) { + expect(files.has(`${dependency.machineName}-${dependency.majorVersion}.${dependency.minorVersion}/library.json`)).toBe(true) + } + for (const [name, json] of files) { + if (!name.endsWith('/library.json')) continue + const library = JSON.parse(json) + expect(library.machineName).not.toMatch(/H5PEditor/) + for (const dependency of [...(library.preloadedDependencies ?? []), ...(library.dynamicDependencies ?? [])]) { + const available = manifest.preloadedDependencies.find((candidate: { machineName: string, majorVersion: string, minorVersion: string }) => + candidate.machineName === dependency.machineName && Number(candidate.majorVersion) === Number(dependency.majorVersion) && Number(candidate.minorVersion) >= Number(dependency.minorVersion)) + expect(available, `${name}: ${dependency.machineName}`).toBeDefined() + } + } + }) + + it('builds standalone Multiple Choice, escapes markup, and preserves false and zero values', async () => { + const path = join(dir, 'single.h5p') + const source = { + ...example, kind: 'multiple-choice', passPercentage: 0, + questions: [{ ...example.questions[0], prompt: ' & "quoted"\nNext line' }] + } + const { manifest, content, libraries } = await buildQuiz(source, path, project) + expect(manifest.mainLibrary).toBe('H5P.MultiChoice') + expect(content.question).toBe('

<script>run()</script> & "quoted"
Next line

') + expect(content.behaviour).toMatchObject({ type: 'single', passPercentage: 0 }) + expect(content.answers).toEqual(expect.arrayContaining([expect.objectContaining({ correct: false })])) + expect(libraries.some(name => name.startsWith('H5P.QuestionSet-'))).toBe(false) + await archive(path) + }) + + it('builds the eight-scenario example without losing questions, feedback, or its pass threshold', async () => { + const path = join(dir, 'scenarios.h5p') + await buildQuiz(scenarios, path, project) + const files = await archive(path) + const content = JSON.parse(files.get('content/content.json')!) + expect(content.passPercentage).toBe(80) + expect(content.questions).toHaveLength(8) + expect(content.questions.filter((question: { params: { behaviour: { type: string } } }) => question.params.behaviour.type === 'multi')).toHaveLength(4) + expect(content.questions[0].params.question).toContain('"Benefits & Payroll:') + expect(content.questions[0].params.question).toContain('
') + for (let i = 0; i < scenarios.questions.length; i++) { + const written = scenarios.questions[i] + const generated = content.questions[i] + expect(generated.metadata.title).toBe(written.title) + expect(generated.params.answers).toHaveLength(4) + for (let j = 0; j < written.answers.length; j++) { + expect(written.answers[j].feedback).toBeTruthy() + expect(generated.params.answers[j].correct).toBe(written.answers[j].correct) + expect(generated.params.answers[j].tipsAndFeedback.chosenFeedback).toBeTruthy() + } + } + }) + + it('does not overwrite an output, publish partial failures, or leave staging directories', async () => { + const path = join(dir, 'existing.h5p') + await writeFile(path, 'keep me') + await expect(buildQuiz(example, path, project)).rejects.toMatchObject({ code: 'EEXIST' }) + expect(await readFile(path, 'utf8')).toBe('keep me') + expect((await readdir(dir)).filter(name => name.startsWith('.h5p-author-'))).toEqual([]) + await expect(buildQuiz({ ...example, questions: [] }, join(dir, 'invalid.h5p'), project)).rejects.toThrow('quiz.questions') + expect(await readdir(dir)).not.toContain('invalid.h5p') + }) + + it('refuses a damaged library part before publishing an archive', async () => { + const damaged = join(dir, 'damaged-pack') + await copyDependencies(damaged) + const library = pack.libraries.find(library => library.machineName === 'H5P.MultiChoice' && library.majorVersion === 1)! + const part = join(damaged, `node_modules/@missing-elements/h5p-libraries/parts/${library.folder}.h5p`) + const bytes = await readFile(part) + const reader = new ZipReader(new BlobReader(new Blob([bytes]))) + try { + const entry = (await reader.getEntries()).find(entry => !entry.directory && entry.filename.endsWith('.js'))! + const offset = entry.offset + 30 + bytes.readUInt16LE(entry.offset + 26) + bytes.readUInt16LE(entry.offset + 28) + // Damage script data without changing library.json, semantics, or the central directory. + bytes[offset + Math.floor(entry.compressedSize / 2)] ^= 1 + } finally { await reader.close() } + await writeFile(part, bytes) + const output = join(damaged, 'quiz.h5p') + await expect(buildQuiz(example, output, damaged)).rejects.toThrow(`Library part failed SHA-256: ${library.folder}`) + expect(await readdir(damaged)).not.toContain('quiz.h5p') + expect((await readdir(damaged)).filter(name => name.startsWith('.h5p-author-'))).toEqual([]) + }) + + it('stops with the install command when the project has an older zip.js', async () => { + const old = join(dir, 'old-zip') + const installed = join(old, 'node_modules/@zip.js/zip.js') + await mkdir(installed, { recursive: true }) + // 2.15 has appendZip without its filter option; only the version is read before the refusal. + await writeFile(join(installed, 'package.json'), JSON.stringify({ name: '@zip.js/zip.js', version: '2.15.0', exports: { './package.json': './package.json' } })) + const error = await buildQuiz(example, join(old, 'quiz.h5p'), old).catch(error => error) + expect(error.message).toContain('Run: npm install --save-dev @missing-elements/h5p-libraries@^1 @zip.js/zip.js@^2.24') + expect(error.cause.message).toBe('@zip.js/zip.js 2.15.0 is older than 2.24') + expect(await readdir(old)).toEqual(['node_modules']) + }) + + it('runs a copied skill outside the checkout and resolves dependencies from the calling project', async () => { + const portable = join(dir, 'portable') + await cp(skill, join(portable, 'installed-skill'), { recursive: true }) + expect(await readFile(join(portable, 'installed-skill/LICENSE'), 'utf8')).toBe(await readFile(resolve(project, '../../LICENSE'), 'utf8')) + await writeFile(join(portable, 'package.json'), '{"private":true}') + await copyDependencies(portable) + const script = join(portable, 'installed-skill/scripts/build-quiz.mjs') + const { stdout } = await exec(process.execPath, [script, join(portable, 'installed-skill/assets/quiz.json'), 'portable.h5p'], { cwd: portable }) + expect(stdout).toContain('Not verified') + const files = await archive(join(portable, 'portable.h5p')) + expect(JSON.parse(files.get('h5p.json')!).mainLibrary).toBe('H5P.QuestionSet') + const failed = exec(process.execPath, [script, join(portable, 'installed-skill/assets/quiz.json'), 'missing.h5p'], { + cwd: dir, env: { ...process.env, NODE_PATH: '' } + }) + await expect(failed).rejects.toMatchObject({ code: 1, stderr: expect.stringContaining('npm install --save-dev') }) + }) +}) diff --git a/docs/verify.md b/docs/verify.md index e032a09a..ba6d3949 100644 --- a/docs/verify.md +++ b/docs/verify.md @@ -16,3 +16,12 @@ With `--libraries pack`, or a copy of the pack's `parts/`, it plays a library th at the pack's newer version, as a site with the pack will, and says which. It is meant as the last step of a pipeline that generates or rewrites packages; the skill in `skills/h5p-verify/` tells an agent when to run it and how to read the result. + +For authoring, the +[`h5p-author` skill](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) +adds a prompt-to-quiz workflow and a portable packaging helper for text-only Multiple Choice +and Question Set. It takes a small quiz JSON file, fills the installed libraries' defaults, +includes their runtime dependencies, then requires this playback check and screenshot review. +The source JSON remains editable; the `.h5p` and full revision from the verification report +identify what was checked. Answer accuracy and a complete question-by-question interaction +check remain separate from the verifier's startup verdict. diff --git a/packages/verify/tests/author.test.mjs b/packages/verify/tests/author.test.mjs new file mode 100644 index 00000000..ce6dbead --- /dev/null +++ b/packages/verify/tests/author.test.mjs @@ -0,0 +1,49 @@ +import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { afterAll, beforeAll, expect, it } from 'vitest' +import { buildQuiz } from '../../../skills/h5p-author/scripts/build-quiz.mjs' +import { verifyPackage } from '../lib/verify.mjs' + +const project = resolve(import.meta.dirname, '../../../apps/demo') +const parts = resolve(import.meta.dirname, '../../libraries/parts') +const example = JSON.parse(await readFile(resolve(import.meta.dirname, '../../../skills/h5p-author/assets/quiz.json'), 'utf8')) +const scenarios = JSON.parse(await readFile(resolve(import.meta.dirname, '../../../skills/h5p-author/assets/phishing-scenarios.json'), 'utf8')) +let dir +const files = new Map() + +beforeAll(async () => { + dir = await mkdtemp(join(tmpdir(), 'h5p-author-playback-')) + for (const kind of ['multiple-choice', 'question-set']) { + const source = { ...example, kind, questions: kind === 'multiple-choice' ? [example.questions[0]] : example.questions } + const file = join(dir, `${kind}.h5p`) + await buildQuiz(source, file, project) + files.set(kind, file) + } + const file = join(dir, 'scenarios.h5p') + await buildQuiz(scenarios, file, project) + files.set('scenarios', file) +}) +afterAll(async () => { await rm(dir, { recursive: true, force: true }) }) + +it.each([ + ['multiple-choice', false], + ['multiple-choice', true], + ['question-set', false], + ['question-set', true], + ['scenarios', false], + ['scenarios', true] +])('plays authored %s (untrusted: %s) without warnings', async (kind, untrusted) => { + const report = await verifyPackage({ + file: files.get(kind), + untrusted, + ...(untrusted ? { libraries: parts } : {}), + settle: 500 + }) + expect(report.verdict, JSON.stringify(report)).toBe('pass') + expect(report.warnings).toEqual([]) + expect(report.errors).toEqual([]) + expect(report.state).toBe('ready') + expect(report.revision).toMatch(/^sha256:[0-9a-f]{64}(; libraries sha256:[0-9a-f]{64})?$/) + expect(report.rendered.text).toBeGreaterThan(100) +}, 60_000) diff --git a/skills/h5p-author/LICENSE b/skills/h5p-author/LICENSE new file mode 100644 index 00000000..c4bf6f37 --- /dev/null +++ b/skills/h5p-author/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 missing-elements + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/skills/h5p-author/SKILL.md b/skills/h5p-author/SKILL.md new file mode 100644 index 00000000..d9a348b1 --- /dev/null +++ b/skills/h5p-author/SKILL.md @@ -0,0 +1,123 @@ +--- +name: h5p-author +description: Create an H5P quiz from a prompt, lesson, or supplied source material. Use when asked to generate, author, or build a playable .h5p Multiple Choice question or Question Set with AI. Includes a portable packaging helper, a quiz example, and the authoring-and-verification loop; no H5P server or provider API needed. +license: MIT +--- + +# Author and verify an H5P quiz + +Turn source material into a playable `.h5p`, not just JSON that looks plausible. +This first workflow supports **text-only H5P.MultiChoice and H5P.QuestionSet**. +It does not implement other content types, media, or a general H5P editor. Do not +quietly substitute a quiz if the user requested another kind of activity. + +Works with Claude Code and other agents that can read skills, write files, and run +Node commands. Ordinary chat without filesystem and command access cannot run the +workflow by itself. No AI provider SDK, API key, MCP server, or repository checkout +is required. Model calls are made by the user's agent, not by the player. +Keep [LICENSE](./LICENSE) with the installed skill and helper. Each sample's +content license is declared separately in its source JSON. + +## 1. Establish the brief + +Use what the user already supplied. Clarify missing essentials: source material, +audience, learning objective, question count, single- or multiple-answer questions, +language, authorship/license, and destination. Do not invent factual source material, +claim expert review, or assign a license the user has not authorized. Use `U` +(undisclosed) when the license is unknown, and ask for the author's name. + +Read [the input format](./references/quiz-format.md) and +[the worked quiz](./assets/quiz.json). Write a source JSON file in the user's +project. These strings are **plain text**, not HTML. The helper escapes them. +For a fuller example, read [the eight-scenario brief and answer key](./references/scenario-quiz.md) +with [its source JSON](./assets/phishing-scenarios.json): mixed answer modes, +explanatory feedback, an 80% threshold, and complete fail/retry/pass checks. + +For each question: + +- Test the stated objective with an answer supported by the supplied material. +- Write plausible distractors and explanatory feedback for every answer. +- Mark exactly one correct answer unless `multipleAnswers` is true. +- For multiple answers, say "Select all that apply" and mark at least one correct. +- Keep the source JSON as the editable deliverable; do not edit generated archives. + +The library defaults, including their Check/Retry/result labels, are English. +`language` records the content language; it does not translate those defaults. +For a localized interface or a feature this helper does not support, explain the +limit rather than delivering an English interface as a translated activity. + +## 2. Prepare the project + +Needs **Node 22 or later**. Run from a writable project directory with a +`package.json` (create one with the project's package manager if needed): + +```bash +npm install --save-dev @missing-elements/h5p-libraries@^1 @zip.js/zip.js@^2.24 +``` + +Use the project's existing package manager if different, and keep its lockfile: +the installed library pack determines the exact library versions in the output. +If either package is already installed at an older version, the helper stops and +prints this command. +The pack contains third-party libraries under their own licenses; preserve their +files/notices and review those terms before distribution. + +Resolve `` below to **this installed skill's directory**, not a path in the +player's repository. Quote paths that contain spaces. Dependencies are resolved +from the project's current working directory, so a globally installed skill works +too. The helper reads the installed pack locally; no hub or CDN is contacted. + +## 3. Build + +```bash +node "/scripts/build-quiz.mjs" quiz.json course.h5p +``` + +The helper validates the source, reads defaults from the installed libraries' +`semantics.json`, assigns each sub-question a UUID, and writes a self-contained +archive with the required runtime dependency closure, not the entire pack. Library +parts are checked against the pack index's SHA-256 before being copied; a mismatch +stops the build and names the part. Library folders stay together before `content/`. +Existing output is never overwritten; use a new filename for a revision. + +There is no media in this workflow, so no normalization step is needed. If a later +workflow adds media, use `h5p-normalize` and verify the final rewritten file. + +## 4. Verify the actual output + +```bash +npx @missing-elements/h5p-verify@latest course.h5p +``` + +Needs Chrome/Edge or Playwright Chromium (`npx playwright install chromium`). +Keep `@latest`: an older Node can otherwise select an old verifier silently. +For a destination that plays uploads untrusted against its pack, also run: + +```bash +npx @missing-elements/h5p-verify@latest course.h5p --untrusted --libraries pack +``` + +Use `--libraries /path/to/parts` to check offline against a specific local pack. +Read `report.json` and open `screenshot.png` in the reported output folder. An exit +code of zero is not enough: read warnings and check the picture. The screenshot +shows startup, not every question or the final result. + +If `h5p-verify` is installed as a skill, follow its detailed troubleshooting loop. +Fix the source or helper, rebuild to a new output path, and verify again. Never +make verification pass by removing a question, changing the learning objective, +or hiding an error. If a browser cannot run, report the package as **unverified**. + +## 5. Review and hand over + +- Check the answer key and feedback against the source, not the verifier's verdict. +- Open the activity in the intended player, answer every question, and inspect + feedback, navigation, score, retry, and completion. With browser tools, record + this evidence; without them, explicitly leave this check to the user. +- A verifier pass means startup/rendering only. Do not claim pedagogical accuracy, + accessibility conformance, full interaction testing, or compatibility with every + LMS. The pack's versions may be newer than those installed in the destination. +- Deliver the `.h5p`, source JSON, the verification report and screenshot, and the + **full `revision` from the report**, with any remaining warnings and limits. +- For a web preview, use `h5p-player-setup`. Prefer `untrusted` with a host-chosen + pack on sites with accounts or packages from other authors. No player page, + deployment, results collection, or AI calls are added by this helper. diff --git a/skills/h5p-author/assets/phishing-scenarios.json b/skills/h5p-author/assets/phishing-scenarios.json new file mode 100644 index 00000000..2b22ecff --- /dev/null +++ b/skills/h5p-author/assets/phishing-scenarios.json @@ -0,0 +1,222 @@ +{ + "kind": "question-set", + "title": "Inbox decisions: a phishing response simulation", + "language": "en", + "license": "CC0 1.0", + "authors": [{ "name": "missing-elements", "role": "Author" }], + "passPercentage": 80, + "questions": [ + { + "title": "An urgent benefits deadline", + "prompt": "You receive: \"Benefits & Payroll: your access ends in 30 minutes. Sign in using this link.\"\nThe demo policy says to open account services through a known bookmark or official app, not an unexpected message's link.\nWhat should you do first?", + "answers": [ + { + "text": "Use the email's link because the deadline is close.", + "correct": false, + "feedback": "Urgency does not change the verification rule. The email is asking you to use a route you have not established as trustworthy." + }, + { + "text": "Reply with your password so Payroll can restore access.", + "correct": false, + "feedback": "Do not send passwords in a reply. The demo policy uses an established route to the service, not a message asking for credentials." + }, + { + "text": "Open the benefits service using your existing bookmark or official app.", + "correct": true, + "feedback": "A route you already know lets you check the account without relying on the unexpected message's link." + }, + { + "text": "Forward the link to a colleague and ask them to try signing in.", + "correct": false, + "feedback": "Asking someone else to try the unverified link transfers the risk instead of verifying the request." + } + ] + }, + { + "title": "Evidence, not appearance", + "prompt": "A message has a familiar logo, correct spelling, a request to send a password, and a threat that your account will close in ten minutes.\nFor this exercise, credential requests and pressure to bypass normal checks are warning signs; branding and spelling do not establish authenticity.\nWhich TWO features are warning signs? Select all that apply.", + "multipleAnswers": true, + "answers": [ + { + "text": "The request to send a password.", + "correct": true, + "feedback": "A request to disclose a password is a reason to stop and verify through the established support channel." + }, + { + "text": "The familiar logo.", + "correct": false, + "feedback": "A logo can be copied. Its presence alone neither proves authenticity nor establishes that a message is fraudulent." + }, + { + "text": "The correct spelling.", + "correct": false, + "feedback": "Fraudulent messages can be well written. Correct spelling alone does not tell you who sent a message." + }, + { + "text": "The pressure to act before the account supposedly closes.", + "correct": true, + "feedback": "Pressure to skip ordinary checks is a warning sign, not a reason to trust the request." + } + ] + }, + { + "title": "A supplier changes bank details", + "prompt": "An email in an existing invoice conversation asks you to change the supplier's bank details. It includes a new telephone number for confirmation.\nThe demo payment policy requires a callback using the supplier number already held in the approved vendor record.\nWhich action follows that policy?", + "answers": [ + { + "text": "Call the supplier using the number in the approved vendor record before changing anything.", + "correct": true, + "feedback": "An independently held contact route checks the change without trusting contact details supplied by the request itself." + }, + { + "text": "Call the new telephone number written in the email.", + "correct": false, + "feedback": "The new number is part of the unverified request. It is not the independent contact route required by the demo policy." + }, + { + "text": "Accept the change because it appears in an existing email conversation.", + "correct": false, + "feedback": "An existing conversation does not replace the required independent callback for a payment-detail change." + }, + { + "text": "Send a small payment to the new account to see whether it arrives.", + "correct": false, + "feedback": "A test transfer is still a transfer to unverified details. It does not satisfy the callback rule." + } + ] + }, + { + "title": "A sign-in approval you did not request", + "prompt": "Your phone asks you to approve a work-account sign-in, but you are not signing in on any device.\nThe demo account policy says to reject unsolicited approvals and report them through the established security channel.\nWhich TWO actions follow the policy? Select all that apply.", + "multipleAnswers": true, + "answers": [ + { + "text": "Approve it once so the repeated notifications stop.", + "correct": false, + "feedback": "Approving an unrequested sign-in can authorize access you did not initiate. Repeated prompts are not a reason to approve." + }, + { + "text": "Reject the sign-in request.", + "correct": true, + "feedback": "Rejecting the request avoids approving a sign-in you did not initiate." + }, + { + "text": "Contact the established security channel to report the unsolicited approval request.", + "correct": true, + "feedback": "Reporting through the established channel lets the team investigate without trusting a contact route supplied by the unexpected request." + }, + { + "text": "Send the approval code to someone who calls claiming to be support.", + "correct": false, + "feedback": "An unexpected caller has not been verified. Sharing an approval code can defeat the account's sign-in protection." + } + ] + }, + { + "title": "The QR-code shortcut", + "prompt": "An unexpected email shows a QR code and says: \"Scan on your phone to bypass the usual sign-in checks.\"\nThe demo rule applies equally to links and QR codes: use the known service route to check an unexpected account request.\nWhat is the best next step?", + "answers": [ + { + "text": "Scan it because a QR code is safer than a clickable link.", + "correct": false, + "feedback": "A QR code is another way to provide a destination. Its format does not establish that the destination or request is trustworthy." + }, + { + "text": "Use a personal phone so the work account cannot be affected.", + "correct": false, + "feedback": "Changing devices does not make an unverified sign-in request trustworthy. You could still disclose work-account credentials." + }, + { + "text": "Ask a colleague to scan it first.", + "correct": false, + "feedback": "Passing the request to a colleague does not verify it and may expose another person to the same risk." + }, + { + "text": "Open the service through the known bookmark or official app instead.", + "correct": true, + "feedback": "The same independent-route rule applies whether an unexpected message supplies a link or a QR code." + } + ] + }, + { + "title": "After entering a password", + "prompt": "You used an email's link and entered your work password. You now suspect the page was false.\nThe demo incident procedure says to report promptly and change an exposed password through the genuine service's known route. Do not send passwords or one-time codes in the report.\nWhich TWO actions follow the procedure? Select all that apply.", + "multipleAnswers": true, + "answers": [ + { + "text": "Report what happened promptly through the established security channel.", + "correct": true, + "feedback": "Prompt reporting gives the team a chance to investigate and contain the incident. Describe the actions you took without sharing secrets." + }, + { + "text": "Wait until you can prove that someone used the account.", + "correct": false, + "feedback": "The demo procedure calls for a prompt report of suspected exposure, not a wait for proof of account misuse." + }, + { + "text": "Change the exposed password through the genuine service's known bookmark or official app.", + "correct": true, + "feedback": "Use the genuine service's established route, not the original email link, to change the exposed password." + }, + { + "text": "Include the password and a current one-time code in the report.", + "correct": false, + "feedback": "A report should not disclose authentication secrets. The procedure explicitly excludes passwords and one-time codes." + } + ] + }, + { + "title": "A sender name is not a verification method", + "prompt": "A message displays your manager's name and asks for an urgent purchase outside the usual approval process.\nThe demo purchasing rule requires you to confirm unusual requests using the manager's contact details already held in the staff directory.\nWhat should you do?", + "answers": [ + { + "text": "Make the purchase because the display name matches your manager.", + "correct": false, + "feedback": "A display name is not the independent confirmation required for a request outside the usual process." + }, + { + "text": "Contact your manager through the staff directory details before proceeding.", + "correct": true, + "feedback": "Using contact details you already hold independently checks the request before you bypass the ordinary purchasing process." + }, + { + "text": "Reply to the message and accept any confirmation sent in the same conversation.", + "correct": false, + "feedback": "A reply in the same unverified conversation is not the independent contact route specified by the demo rule." + }, + { + "text": "Use the new contact number supplied in the urgent message.", + "correct": false, + "feedback": "A contact number supplied by the unverified request cannot independently establish who made that request." + } + ] + }, + { + "title": "A useful incident report", + "prompt": "You are reporting a suspicious message through the established security channel.\nThe demo reporting procedure asks for when and where it arrived and what you did, including whether you opened a link or entered credentials. It excludes passwords and one-time codes.\nWhich TWO items belong in that report? Select all that apply.", + "multipleAnswers": true, + "answers": [ + { + "text": "Your current password, so the team can test whether it still works.", + "correct": false, + "feedback": "The report must not contain a password. The team does not need you to disclose it to understand what happened." + }, + { + "text": "A one-time sign-in code copied from your phone.", + "correct": false, + "feedback": "One-time codes are authentication secrets and are explicitly excluded from the demo reporting procedure." + }, + { + "text": "When and where the message arrived, with its sender and subject.", + "correct": true, + "feedback": "These details help identify the message and establish a timeline without disclosing an authentication secret." + }, + { + "text": "Whether you opened a link, approved a sign-in, or entered credentials.", + "correct": true, + "feedback": "A factual description of your actions helps the team assess possible exposure. State that credentials were entered without including their values." + } + ] + } + ] +} diff --git a/skills/h5p-author/assets/quiz.json b/skills/h5p-author/assets/quiz.json new file mode 100644 index 00000000..5301878e --- /dev/null +++ b/skills/h5p-author/assets/quiz.json @@ -0,0 +1,48 @@ +{ + "kind": "question-set", + "title": "Practice: recognizing phishing", + "language": "en", + "license": "CC0 1.0", + "authors": [{ "name": "Example author", "role": "Author" }], + "passPercentage": 100, + "questions": [ + { + "title": "An unexpected sign-in link", + "prompt": "An unexpected email asks you to sign in urgently using its link. What is the safest next step?", + "answers": [ + { + "text": "Open the service using a known bookmark or its official app.", + "correct": true, + "feedback": "Use a route you already trust instead of the link in the unexpected message." + }, + { + "text": "Use the link because the message includes the company's logo.", + "correct": false, + "feedback": "Logos can be copied. Branding alone does not establish that a message is genuine." + } + ] + }, + { + "title": "Warning signs", + "prompt": "Which features of an unexpected email are reasons to verify it independently? Select all that apply.", + "multipleAnswers": true, + "answers": [ + { + "text": "It asks for your password by email.", + "correct": true, + "feedback": "Do not send passwords by email. Check the request through your organization's established channel." + }, + { + "text": "It pressures you to act immediately to avoid losing access.", + "correct": true, + "feedback": "Pressure and urgency can be used to stop you checking a request carefully." + }, + { + "text": "It uses correct spelling.", + "correct": false, + "feedback": "Correct spelling does not prove a message is genuine or fraudulent." + } + ] + } + ] +} diff --git a/skills/h5p-author/references/quiz-format.md b/skills/h5p-author/references/quiz-format.md new file mode 100644 index 00000000..88f87629 --- /dev/null +++ b/skills/h5p-author/references/quiz-format.md @@ -0,0 +1,40 @@ +# Quiz source format + +The helper accepts this small authoring format, not raw H5P `content.json`. +Unknown fields and wrong types fail with a path; nothing is silently dropped. +All authored text is plain text. `