From e0047beb861998296017bda78ae23d21975be12b Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Sun, 1 Nov 2026 02:31:02 +0200 Subject: [PATCH 1/5] feat(h5p-author): any content type in the pack, from a package and a file per type (#100) The h5p-author skill built Multiple Choice and Question Set from a bundled script. It now builds 28 content types: the agent chooses the type with the user, reads one file for it, and writes a source of only what differs from the type's defaults. - packages/author, @missing-elements/h5p-author 0.1.0: `build` and `schema`, the library pack pinned, every written value checked as H5P's editor would, media copied and described, layouts for the types the editor gives a widget of their own (Course Presentation, Interactive Video, Branching Scenario, Game Map, Find the Hotspot). - The sweep's ContentGenerator moved into the package; the sweep runs it lenient with its own fill, and generates the same bytes for all 55 types. - The skill: a router SKILL.md, references/authoring.md, and references/types/.md with notes, a tested example and the fields, generated from the pack and kept current by the weekly refresh. - The quiz format stays a source the build takes. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/content-types.yml | 2 + .github/workflows/libraries.yml | 4 +- .gitignore | 5 +- AGENTS.md | 157 ++++- README.md | 12 +- apps/demo/package.json | 1 + apps/demo/scripts/lib/sweep-content.mjs | 222 +------ apps/demo/scripts/sweep-content-types.mjs | 15 +- apps/demo/tests/sweep-content.test.ts | 98 +-- apps/skills/package.json | 1 + apps/skills/tests/author-playback.test.ts | 11 +- apps/skills/tests/author-verify.test.ts | 177 +++-- docs/development.md | 7 +- docs/verify.md | 13 +- package.json | 1 + packages/author/README.md | 127 ++++ packages/author/bin/h5p-author.mjs | 89 +++ packages/author/lib/author.mjs | 56 ++ packages/author/lib/build.mjs | 383 +++++++++++ packages/author/lib/generator.mjs | 609 ++++++++++++++++++ packages/author/lib/media.mjs | 307 +++++++++ packages/author/lib/pack.mjs | 214 ++++++ packages/author/lib/quiz.mjs | 149 +++++ packages/author/lib/schema.mjs | 224 +++++++ packages/author/lib/text.mjs | 159 +++++ packages/author/lib/translate.mjs | 123 ++++ packages/author/lib/types.mjs | 493 ++++++++++++++ packages/author/package.json | 66 ++ .../author/scripts/check-pack-published.mjs | 26 + packages/author/scripts/sync-skill.mjs | 27 + .../author/tests/build.test.ts | 386 +++++++---- packages/author/tests/generator.test.ts | 232 +++++++ packages/author/tests/layout.test.ts | 227 +++++++ packages/author/tests/package-types.ts | 19 + packages/author/tests/text.test.ts | 108 ++++ packages/author/tsconfig.json | 17 + packages/author/tsconfig.types.json | 12 + pnpm-lock.yaml | 28 + skills/h5p-author/SKILL.md | 212 +++--- skills/h5p-author/references/authoring.md | 46 ++ skills/h5p-author/references/quiz-format.md | 26 +- skills/h5p-author/references/scenario-quiz.md | 11 +- .../references/types/H5P.Accordion.md | 32 + .../references/types/H5P.AdvancedText.md | 28 + .../h5p-author/references/types/H5P.Audio.md | 29 + .../h5p-author/references/types/H5P.Blanks.md | 53 ++ .../references/types/H5P.BranchingScenario.md | 75 +++ .../h5p-author/references/types/H5P.Column.md | 33 + .../types/H5P.CoursePresentation.md | 91 +++ .../references/types/H5P.Dialogcards.md | 49 ++ .../references/types/H5P.Dictation.md | 64 ++ .../references/types/H5P.DragText.md | 44 ++ .../h5p-author/references/types/H5P.Essay.md | 69 ++ .../references/types/H5P.GameMap.md | 160 +++++ .../h5p-author/references/types/H5P.Image.md | 32 + .../types/H5P.ImageHotspotQuestion.md | 53 ++ .../references/types/H5P.ImageHotspots.md | 50 ++ .../references/types/H5P.ImagePair.md | 37 ++ .../references/types/H5P.ImageSequencing.md | 40 ++ .../references/types/H5P.InteractiveBook.md | 50 ++ .../references/types/H5P.InteractiveVideo.md | 110 ++++ .../references/types/H5P.MarkTheWords.md | 41 ++ .../references/types/H5P.MemoryGame.md | 45 ++ .../references/types/H5P.MultiChoice.md | 61 ++ .../references/types/H5P.MultiMediaChoice.md | 53 ++ .../references/types/H5P.QuestionSet.md | 98 +++ .../references/types/H5P.Questionnaire.md | 57 ++ .../references/types/H5P.SingleChoiceSet.md | 44 ++ .../references/types/H5P.SortParagraphs.md | 48 ++ .../references/types/H5P.Summary.md | 38 ++ .../h5p-author/references/types/H5P.Text.md | 28 + .../references/types/H5P.Timeline.md | 63 ++ .../references/types/H5P.TrueFalse.md | 45 ++ .../h5p-author/references/types/H5P.Video.md | 42 ++ skills/h5p-author/scripts/build-quiz.mjs | 446 ------------- 75 files changed, 6181 insertions(+), 1129 deletions(-) create mode 100644 packages/author/README.md create mode 100755 packages/author/bin/h5p-author.mjs create mode 100644 packages/author/lib/author.mjs create mode 100644 packages/author/lib/build.mjs create mode 100644 packages/author/lib/generator.mjs create mode 100644 packages/author/lib/media.mjs create mode 100644 packages/author/lib/pack.mjs create mode 100644 packages/author/lib/quiz.mjs create mode 100644 packages/author/lib/schema.mjs create mode 100644 packages/author/lib/text.mjs create mode 100644 packages/author/lib/translate.mjs create mode 100644 packages/author/lib/types.mjs create mode 100644 packages/author/package.json create mode 100644 packages/author/scripts/check-pack-published.mjs create mode 100644 packages/author/scripts/sync-skill.mjs rename apps/skills/tests/author.test.ts => packages/author/tests/build.test.ts (50%) create mode 100644 packages/author/tests/generator.test.ts create mode 100644 packages/author/tests/layout.test.ts create mode 100644 packages/author/tests/package-types.ts create mode 100644 packages/author/tests/text.test.ts create mode 100644 packages/author/tsconfig.json create mode 100644 packages/author/tsconfig.types.json create mode 100644 skills/h5p-author/references/authoring.md create mode 100644 skills/h5p-author/references/types/H5P.Accordion.md create mode 100644 skills/h5p-author/references/types/H5P.AdvancedText.md create mode 100644 skills/h5p-author/references/types/H5P.Audio.md create mode 100644 skills/h5p-author/references/types/H5P.Blanks.md create mode 100644 skills/h5p-author/references/types/H5P.BranchingScenario.md create mode 100644 skills/h5p-author/references/types/H5P.Column.md create mode 100644 skills/h5p-author/references/types/H5P.CoursePresentation.md create mode 100644 skills/h5p-author/references/types/H5P.Dialogcards.md create mode 100644 skills/h5p-author/references/types/H5P.Dictation.md create mode 100644 skills/h5p-author/references/types/H5P.DragText.md create mode 100644 skills/h5p-author/references/types/H5P.Essay.md create mode 100644 skills/h5p-author/references/types/H5P.GameMap.md create mode 100644 skills/h5p-author/references/types/H5P.Image.md create mode 100644 skills/h5p-author/references/types/H5P.ImageHotspotQuestion.md create mode 100644 skills/h5p-author/references/types/H5P.ImageHotspots.md create mode 100644 skills/h5p-author/references/types/H5P.ImagePair.md create mode 100644 skills/h5p-author/references/types/H5P.ImageSequencing.md create mode 100644 skills/h5p-author/references/types/H5P.InteractiveBook.md create mode 100644 skills/h5p-author/references/types/H5P.InteractiveVideo.md create mode 100644 skills/h5p-author/references/types/H5P.MarkTheWords.md create mode 100644 skills/h5p-author/references/types/H5P.MemoryGame.md create mode 100644 skills/h5p-author/references/types/H5P.MultiChoice.md create mode 100644 skills/h5p-author/references/types/H5P.MultiMediaChoice.md create mode 100644 skills/h5p-author/references/types/H5P.QuestionSet.md create mode 100644 skills/h5p-author/references/types/H5P.Questionnaire.md create mode 100644 skills/h5p-author/references/types/H5P.SingleChoiceSet.md create mode 100644 skills/h5p-author/references/types/H5P.SortParagraphs.md create mode 100644 skills/h5p-author/references/types/H5P.Summary.md create mode 100644 skills/h5p-author/references/types/H5P.Text.md create mode 100644 skills/h5p-author/references/types/H5P.Timeline.md create mode 100644 skills/h5p-author/references/types/H5P.TrueFalse.md create mode 100644 skills/h5p-author/references/types/H5P.Video.md delete mode 100644 skills/h5p-author/scripts/build-quiz.mjs diff --git a/.github/workflows/content-types.yml b/.github/workflows/content-types.yml index db936c6a..b18be677 100644 --- a/.github/workflows/content-types.yml +++ b/.github/workflows/content-types.yml @@ -16,6 +16,8 @@ on: - 'apps/demo/sweep/**' - 'apps/demo/scripts/sweep-content-types.mjs' - 'apps/demo/scripts/lib/sweep-content.mjs' + # The generator the sweep runs, the author package's since #100. + - 'packages/author/lib/generator.mjs' - '.github/workflows/content-types.yml' schedule: - cron: '0 7 * * 1' diff --git a/.github/workflows/libraries.yml b/.github/workflows/libraries.yml index 1def6aed..0bffe64a 100644 --- a/.github/workflows/libraries.yml +++ b/.github/workflows/libraries.yml @@ -62,6 +62,8 @@ jobs: # The player's libraries="pack" pins this version and carries its parts' index: stamp # both in the same change. pnpm --filter @missing-elements/h5p-offline-player sync:assets + # The h5p-author skill's type files list each content type's fields from the pack. + pnpm --filter @missing-elements/h5p-author sync:skill # Every content type played against the new pack, for the review: CI does not start on # a pull request a workflow opened, so the result goes into its description. @@ -76,7 +78,7 @@ jobs: git config user.name 'github-actions[bot]' git config user.email '41898282+github-actions[bot]@users.noreply.github.com' git switch -C "$branch" - git add packages/libraries packages/player/src/shared/constants.ts packages/player/src/shared/pack-index.ts + git add packages/libraries packages/player/src/shared/constants.ts packages/player/src/shared/pack-index.ts skills/h5p-author/references/types git commit -m "chore(libraries): refresh from the H5P hub, $version" git push --force origin "$branch" diff --git a/.gitignore b/.gitignore index 0309098c..52072b77 100644 --- a/.gitignore +++ b/.gitignore @@ -1,7 +1,9 @@ node_modules dist dist-demo +# The plain-JS packages' generated declarations; the h5p-author skill's content-type files are not. types +!skills/h5p-author/references/types/ .vercel # Generated: the player's `sync:assets` vendors the h5p-standalone runtime into @@ -12,9 +14,10 @@ public # Copied from the repository root by the player's `prepack`, so npm finds them in the package. packages/player/README.md packages/player/LICENSE -# The same for the normalizer, the verifier, the cmi5 wiring and the embed site writer, whose `prepack` copies the LICENSE. +# The same for the normalizer, the verifier, the author, the cmi5 wiring and the embed site writer, whose `prepack` copies the LICENSE. packages/normalize/LICENSE packages/verify/LICENSE +packages/author/LICENSE packages/cmi5/LICENSE packages/embed/LICENSE diff --git a/AGENTS.md b/AGENTS.md index bfb3e7aa..8ac4bf97 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,8 +28,8 @@ The hard constraints that shape every file here: ## Layout -A pnpm workspace of ten parts: eight packages and two apps, the demo and the skills' test project. -Seven of the packages are published to npm; the eighth, the VS Code extension, is a `.vsix`. Paths +A pnpm workspace of eleven parts: nine packages and two apps, the demo and the skills' test project. +Eight of the packages are published to npm; the ninth, the VS Code extension, is a `.vsix`. Paths elsewhere in this file are relative to the part they belong to — `src/…` and `scripts/…` mean the player's unless a section says otherwise. @@ -167,6 +167,28 @@ packages/verify/ @missing-elements/h5p-verify — plays a package headl ranges, a browser playing from it, its ending and its command line types/ generated, not in git: the declarations, `VerifyReport` among them +packages/author/ @missing-elements/h5p-author — a playable .h5p of any content type in the pack from a short + source, published as a command (see *The author package*) + bin/h5p-author.mjs the command: `build` and `schema` + lib/build.mjs the source read and checked, the libraries loaded and localized, the + content generated, media copied, layouts run, the archive written + lib/generator.mjs `ContentGenerator`: content from a type's `semantics.json`, authoring + (every written value checked as the editor's form would) or lenient, as + the sweep runs it; moved out of the demo on 2026-11-01 (#100) + lib/types.mjs what the editor's own widgets set and check, per content type: Course + Presentation's layout, Interactive Video's placement, Branching + Scenario's paths, Game Map's stages, Find the Hotspot's areas + lib/text.mjs, translate.mjs, media.mjs, pack.mjs, schema.mjs, quiz.mjs text as H5P stores it; the + libraries' translations; files and web addresses; the pinned pack read + by part; the field list a source is written from; the skill's quiz format + scripts/ sync-skill.mjs writes the skill's type files' generated sections from the + pack (`pnpm --filter @missing-elements/h5p-author sync:skill`); + check-pack-published.mjs, the publish guard for the pinned pack + tests/ node tests: the generator, text and translations, layouts and schema, + builds with media from disk and a local server, the command; + package-types.ts as for the normalizer + types/ generated, not in git: the declarations + packages/cmi5/ @missing-elements/h5p-cmi5 — cmi5 for the element, published as a library src/index.ts startCmi5(player): the launch, the outcome, terminate; its pure parts (allowedStatement, cmi5Score, webAddress) exported for tests @@ -249,7 +271,7 @@ apps/demo/ the demo app: the dev server for the whole repository, tests/app.test.ts the installable app's screens, history and keyboard, through the dev server; engine.ts and harness.ts its helpers. The demo's other pages have no tests (see *Commands*) - tests/sweep-content.test.ts the sweep's content generator, in Node + tests/sweep-content.test.ts the sweep's fill for h5p-author's generator, in Node cmi5-catapult/ docker-compose.yml for ADL's CATAPULT player, MySQL and the SQL LRS; scripts/cmi5-catapult.mjs clones CATAPULT into it (ignored) and proves the cmi5 page against the player @@ -264,7 +286,8 @@ apps/demo/ the demo app: the dev server for the whole repository, scripts/ build-demo (the site into dist-demo/), build-demo-content, build-og-image, build-app-icons, build-app-screenshots, build-libraries (the pack, into packages/libraries), sweep-content-types (every content type in the pack - played; see *The content-type sweep*), lib/sweep-content.mjs its generator, + played; see *The content-type sweep*), lib/sweep-content.mjs its way of running + h5p-author's generator, lib/code-blocks.mjs the Vite plugin that highlights the pages' code, lib/hub.mjs the hub host, the cache and the download the two hub-reading scripts share, lib/pages.mjs the one list of the site's pages, for the @@ -281,20 +304,15 @@ apps/demo/ the demo app: the dev server for the whole repository, site root -> dist-demo/ apps/skills/ the skills' test project, a project of the kind a skill's user runs it in: - it installs what h5p-author tells its user to install, the library pack and - zip.js, which the script resolves from it, and the verifier the skill runs - with npx; nothing in it ships. Out of the demo since 2026-10-23, by the - owner's choice - tests/author.test.ts h5p-author's helper in Node: its quiz validation, its archives, the - interface texts from the libraries' translations for the quiz's language, - a damaged library part refused by its SHA-256 before anything is - published, a zip.js older than 2.24 stopped with the install command, - and the skill copied outside the checkout, resolving its dependencies - from the calling project - tests/author-verify.test.ts both quiz kinds, the eight-scenario example, a quiz in Ukrainian and one - whose titles and author carry quotes and markup through the verifier's - published API, trusted and untrusted against the installed pack: each a - pass with no warnings + it installs the commands the skills run with npx, the author package and + the verifier, and the library pack; nothing in it ships. Out of the demo + since 2026-10-23, by the owner's choice + tests/author-verify.test.ts every h5p-author type file's example built, with sample media, and + played trusted and untrusted through the verifier's published API, each + a pass with no warnings; each file's generated section compared with + the pack's, so a pack refresh shows a change; and the quiz format: both + kinds, the eight-scenario example, a quiz in Ukrainian and one whose + titles and author carry quotes and markup tests/author-playback.test.ts the eight-scenario example in the verifier's preview, the review the skill hands a person, trusted and untrusted: every question's feedback, 75% failing and 83.3% passing an 80% threshold, retry to 100%, and only @@ -325,12 +343,12 @@ skills/ agent skills, one directory per skill, at the root bec `npx skills add ` looks: h5p-verify (when to run the verifier and how to read its report), h5p-player-setup (putting the player on a site), h5p-normalize (when a package needs rewriting to stream, and how to read the dry run) - and h5p-author (text-only Multiple Choice and Question Set from a prompt: - a quiz JSON source, the installed pack's semantics, its translations for - the quiz's language (2026-10-31, #96) and runtime dependency closure, - then the verifier's check, its screenshot and its preview for the user; - its bundled scripts/build-quiz.mjs resolves dependencies from the - caller's project, and is tested from apps/skills) + and h5p-author (any of 28 content types from a prompt, since #100: the + type chosen with the user first, a source of only what differs from the + defaults, written from references/types/.md — notes, a tested + example and the fields, their section generated from the pack — built + with `npx @missing-elements/h5p-author`, then the verifier's check, its + screenshot and its preview for the user; tested from apps/skills) api/no-range.js the Vercel function that stands in for a host without Range on the demo vercel.json the deployment: build command, the /no-range rewrite, caching and security headers @@ -369,6 +387,8 @@ pnpm typecheck # every package pnpm build # the player: types, element, both workers; builds the runtime package on the way pnpm normalize course.h5p # rewrite a package so it streams: media stored, mp4 index first pnpm verify course.h5p # play a package in a headless browser and report whether it works +pnpm author build source.json course.h5p # a package of any content type in the pack from a short source +pnpm --filter @missing-elements/h5p-author sync:skill # the h5p-author skill's type files' fields, from the pack pnpm --filter h5p-preview package # the VS Code extension's .vsix, from the player as pnpm build left it pnpm --filter h5p-preview test:vscode # its suite in a VS Code it downloads; under xvfb-run on Linux pnpm --filter h5p-preview screenshot # its listing's screenshot, images/preview.png, in that VS Code @@ -438,10 +458,11 @@ cmi5 page also against CATAPULT (`pnpm cmi5:catapult`). Where a page's test was package's behaviour, the test moved into the package: the embed page asking before it plays a package that needs its own code, and `resizer.js` answering a frame that loaded before it, are in `packages/embed/tests/site.test.ts`; the cmi5 page's were already in `packages/cmi5/tests/`. -The skills' scripts are source too, tested from `apps/skills`, a project of their own (2026-10-23, -the owner's choice). h5p-author's tests were first in the demo, which had the pack and zip.js -installed, and in the verifier's tests, and the playback test played the quiz through the -installable app. It plays it in the verifier's preview now, the page the skill hands a person to +The skills are source too, tested from `apps/skills`, a project of their own (2026-10-23, the +owner's choice): what they tell an agent to write builds and plays. h5p-author's own code is the +author package's since #100, tested with it. Its tests were first in the demo, which had the pack +and zip.js installed, and in the verifier's tests, and the playback test played the quiz through +the installable app. It plays it in the verifier's preview now, the page the skill hands a person to review it in, and in Chromium only, where in the demo the `engines` job ran it in Firefox and WebKit too; the player's own suite covers the engines. @@ -1928,6 +1949,79 @@ The tests need the player built (`pnpm build`); the package's `test` script does The repository's own sweep does click through content and read its statements (next section), for packages it makes; the verifier, which is handed a stranger's, still does not. +## The author package + +`@missing-elements/h5p-author` builds a playable package of any content type in the pack from a +source of only what an author would write: `{ type, title, language, license, authors, params }`, +`params` holding the fields that differ from their defaults. Built on 2026-11-01 for #100, the +owner's design: the content type chosen with the user first, a file per type in the h5p-author +skill so no agent works out how a type is built, a short skill, one package for the whole pack. It +replaced the skill's bundled `scripts/build-quiz.mjs`, which built Multiple Choice and Question Set +alone, and its quiz format stays a source the build takes (`lib/quiz.mjs` maps it), so nothing +written in it broke. Measured on a five-question quiz, at four characters a token: `content.json` +was about 3,470 tokens to write by hand, after reading 3,900 to 9,500 of a type's semantics; the +source is 1,000 to 1,800 tokens, written from a type file of 200 to 3,000. Six things the code does +not say by itself: + +- **One generator, two ways to run it.** `ContentGenerator` was the sweep's + (`apps/demo/scripts/lib/sweep-content.mjs`), and moved here so there is one copy; the sweep runs + it `lenient`, keeping what its files write as written and filling every required field from its + own samples, and its 55 types came out byte for byte the same after the move. Authoring checks + every written value as H5P's editor would let an author save it, and reports every problem with + its path, all at once, so a fix is one round: a field the semantics do not name, a select's + unknown option, a number out of range or with decimals the editor's number field refuses, a list + too short or long, a required field with neither a value nor a default, sub-content of a library + the field does not take. What the editor does is the rule, read from h5p-editor-php-library's + widgets on 2026-11-01: a new list starts with `defaultNum`, `min` or one item, and a list such an + item needs writing in is reported as one missing list; a field its `showWhen` rule hides is not + asked for (Multimedia Choice's poster, wanted only for audio); media fields, which the editor + never asks for, are required where the semantics say so, and optional where a type plays without + them (H5P.Video's poster); a library default nobody opened, Interactive Video's summary task, is + left as the editor leaves it, asking nothing. +- **Text is stored as the platforms store it.** A plain field is escaped as `htmlspecialchars` + does, existing entities kept, and a tag in it is noted, since a learner sees it as written. A + field with `tags` takes markup with the tags the content check keeps for it (its own, `div`, + `span`, `p`, `br`, and what they imply), checked here rather than filtered later: any other tag, + an event handler, a style the field does not allow, a script address or a comment is a problem + by name. Plain text written into such a field is wrapped as the editor wraps typing: in `

` + where the field's `enterMode` is `p` or its tags offer headings, `pre`, `p` or `div`, in `

` + otherwise. Every plain default is escaped too, English or translated, and a `null` default (Course + Presentation's colours) is left out, which the content check would otherwise turn into a + warning. Every type file's example plays untrusted with no warning, which is the test of this. +- **Some types need what their editor widget does.** Course Presentation, Interactive Video, Game + Map and Branching Scenario are edited on a slide, a video, a map and a tree of their own, which set + fields the form hides (`widget: "none"`). `lib/types.mjs` does each one's work on the generated + content, wherever the type appears: a slide's unplaced elements stacked, text as tall as its lines + on CP's 16:9 slide at 16 px per 640, pictures shrunk in shape, a question in what is left, with a + note when a slide holds too much; interactions placed apart and named by their library; stages + given ids and types, neighbours named by label made indices both ways; end screens numbered. It + checks what those widgets would not let pass: a scenario node with no next, a link to no node, a + loop no path leaves; a map with no start; a hotspot question with no right area. A hidden field + nothing set is reported. Lists such a widget draws start empty. +- **Media are copied and described.** A file is `{ path }`, beside the source or an `http(s)` + address, downloaded at build time; it lands under `content/images|videos|audios|files/` with its + MIME type, an image's size read from its header (PNG, JPEG, GIF, BMP), and its copyright, on + H5P's own list of types (no SVG, no WebP). A YouTube, Vimeo or Panopto address stays, as the + editor embeds it. A package with media goes through the normalizer's `normalizeArchive`, so it is + stored and laid out to stream, an mp4's index first. An addon whose pattern the text matches, + H5P.MathDisplay for LaTeX, is carried as a site with it installed exports it. +- **The pack is pinned.** The dependency is `workspace:*`, published as the exact version + packages/libraries has, so a release builds with the libraries it was tested with and one command + needs no project set up; `--libraries` names another `parts/` folder. Each part is checked + against the index's SHA-256 before it is read. `prepublishOnly` refuses while that version is not + on npm with this checkout's index (`scripts/check-pack-published.mjs`), as the player's does. +- **The skill's type files are partly generated.** `skills/h5p-author/references/types/.md` + holds hand-written notes, an example, and the type's fields between two markers, written by + `schemaOf(…, { markdown: true })`: names, kinds, defaults, options, list sizes, a markup field's + tags, the sub-content the pack has, the syntax the editor's `important` box explains, `showWhen` + conditions; interface texts and the fields a layout sets left out. The weekly pack refresh runs + `sync:skill`; `apps/skills`' test fails on a file out of date, and builds and plays every + example. 28 content types have files, and the parts other content holds (Text, Image, Video); + any other type's fields are one `h5p-author schema` away. + +Not built: editing an existing package, layout for Drag and Drop and Crossword, and any judgement +of whether content is right, which stays the person's, in the verifier's preview. + ## The VS Code extension `packages/vscode` is **H5P Preview by Missing Elements** (`MissingElements.h5p-preview`): a `.h5p` @@ -2306,7 +2400,7 @@ Not covered: ## Types for the plain-JS packages -The normalizer, the verifier and the embed package are JavaScript with JSDoc, and since #60 their +The normalizer, the verifier, the embed package and the author are JavaScript with JSDoc, and since #60 their documented API ships with declarations generated from it: `tsconfig.json` checks `lib/` (and `site/embed.js`) with `checkJs`, `tsconfig.types.json` emits the entries' declarations into `types/` with `stripInternal`, and each `exports` entry names them under `types`. `pnpm types` @@ -2349,8 +2443,9 @@ been measured on four content types: the hub serves no example content, and h5p. carry licences of their own and old versions, so each package is made here. 110 plays take about three minutes locally, five on CI. -- **Content is generated from the content type's own `semantics.json`** - (`lib/sweep-content.mjs`), as H5P's editor saves a form: every default, and for a required field +- **Content is generated from the content type's own `semantics.json`** by the author package's + generator, run lenient with the sweep's fill (`lib/sweep-content.mjs`; one copy since #100, + checked byte for byte on all 55 types when it moved), as H5P's editor saves a form: every default, and for a required field without one the smallest value the content check accepts — a label as text, the first sample a pattern accepts, the minimum number, the first option, a list as long as its `min` or `defaultNum`, the sample image, video or audio. A group's own default is its whole value; a group diff --git a/README.md b/README.md index 9057b53b..fb96f063 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,10 @@ something else. [privacy](https://github.com/missing-elements/h5p-offline-player/blob/main/docs/privacy.md). - **Comes with tools for pipelines.** [`h5p-verify`](https://github.com/missing-elements/h5p-offline-player/blob/main/docs/verify.md) plays a package headless and says whether it works, [`h5p-normalize`](https://github.com/missing-elements/h5p-offline-player/blob/main/docs/streaming-video.md) - rewrites one so its video streams, and [`h5p-embed`](https://github.com/missing-elements/h5p-offline-player/tree/main/packages/embed#readme) - writes the embed site. The [agent skills](#agent-skills) cover those workflows and quiz authoring. + rewrites one so its video streams, [`h5p-embed`](https://github.com/missing-elements/h5p-offline-player/tree/main/packages/embed#readme) + writes the embed site, and [`h5p-author`](https://github.com/missing-elements/h5p-offline-player/tree/main/packages/author#readme) + builds a package of any content type from a short JSON source. The [agent skills](#agent-skills) + cover those workflows and authoring. - **Is tested where it runs.** CI runs the browser tests in Chromium, Firefox and WebKit on every change, and plays every content type in the library pack, untrusted and trusted, on each change to the player, the runtime or the pack, and every week. @@ -199,8 +201,10 @@ npx skills add missing-elements/h5p-offline-player ``` [`h5p-author`](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) -turns a prompt or lesson into a Multiple Choice or Question Set, verifies it in the real player and -opens it in your browser for you to play before handing it over. `h5p-player-setup` puts H5P on a +turns a prompt or lesson into H5P content of any of 28 types, chosen with you first: quizzes, fill +in the blanks, flash cards, presentations, interactive books, branching scenarios, interactive video +and more. It verifies the package in the real player and opens it in your browser for you to play +before handing it over. `h5p-player-setup` puts H5P on a site, `h5p-normalize` fixes packages whose video will not stream, and `h5p-verify` checks that a generated package plays. They work with Claude Code and other agents with filesystem and command access, with no provider API key or H5P server; a verifier pass checks startup and rendering, not diff --git a/apps/demo/package.json b/apps/demo/package.json index 81770a77..59865fc7 100644 --- a/apps/demo/package.json +++ b/apps/demo/package.json @@ -23,6 +23,7 @@ "@vercel/analytics": "^2.0.1" }, "devDependencies": { + "@missing-elements/h5p-author": "workspace:*", "@missing-elements/h5p-libraries": "workspace:*", "@missing-elements/h5p-normalize": "workspace:*", "@types/node": "^26.6.5", diff --git a/apps/demo/scripts/lib/sweep-content.mjs b/apps/demo/scripts/lib/sweep-content.mjs index df5faa14..a18986d2 100644 --- a/apps/demo/scripts/lib/sweep-content.mjs +++ b/apps/demo/scripts/lib/sweep-content.mjs @@ -1,17 +1,19 @@ +import { ContentGenerator, librariesIn, libraryString } from '@missing-elements/h5p-author' + /** - * Content for a content type, made from its own `semantics.json` as H5P's editor would save a form: - * what is written by hand in `apps/demo/sweep/.json` where it says something, and - * everywhere else every default the semantics give and, for a required field without one, the - * smallest value the content check accepts. A field that holds another content type takes the - * library the hand-written value names, filled in the same way, or else the first of its options - * the pack has. So a hand-written file states only what generation cannot know — a question's - * right answer, which drop zone an item belongs in — and a list in it is as long as it is written. + * The sweep's way of running the content generator, which is @missing-elements/h5p-author's (one + * copy since 2026-11-01, #100): lenient, taking what `apps/demo/sweep/.json` writes as + * it is, and filling every required field the semantics give no default for with the smallest + * value the content check accepts — a label as text, the first sample a pattern accepts, the + * minimum number, the first option, the sample image, video or audio, the first sub-content the + * pack has. Authoring refuses such a field instead; the sweep has to make every content type play. * - * Pure: the pack is handed in as a map of folders, and a value points at media by the paths in - * `MEDIA` without writing them, so `tests/sweep-content.test.ts` runs it in Node. The sweep finds - * the files and the libraries a package uses in its final parameters (`mediaIn`, `librariesIn`). + * Deterministic: sub-content ids are numbered, so generated packages are the same bytes every run. + * `tests/sweep-content.test.ts` holds the sweep's rules; the generator's own are the package's. */ +export { ContentGenerator, librariesIn, libraryString } + /** The files a generated value may point at, under `content/`, and what each one is. */ export const MEDIA = { image: { path: 'images/sample.png', mime: 'image/png', width: 640, height: 360 }, @@ -27,181 +29,28 @@ export const MEDIA = { */ const PATTERNED_SAMPLES = ['Sample', '#808080', 'https://example.org/', 'sample@example.org', '+1 555 0100', '2026/1/1', '12:00', '2026,1,1', '100px', '0', 'a'] -/** The nesting depth at which a library field stops choosing content of its own. */ -const MAX_DEPTH = 4 - -/** @typedef {{ folder: string, library: any, semantics: any[] | null }} PackLibrary */ - -export class ContentGenerator { - /** @param {Map} pack every library folder in the pack, by folder name */ - constructor(pack) { - this.pack = pack - this.ids = 0 - } - - /** - * The newest library in the pack with this machine name and, when given, this major — the - * one the player serves for any minor of it. - * @param {string} machineName - * @param {number} [major] - */ - find(machineName, major) { - let best = null - for (const entry of this.pack.values()) { - const { library } = entry - if (library.machineName !== machineName) continue - if (major !== undefined && library.majorVersion !== major) continue - if (!best || library.majorVersion > best.library.majorVersion || (library.majorVersion === best.library.majorVersion && library.minorVersion > best.library.minorVersion)) best = entry - } - return best - } - - /** - * The parameters for a library's content, with `written` — the hand-written parameters, if - * any — laid in. - * @param {PackLibrary} entry - * @param {any} [written] - */ - generate(entry, written) { - return this.fields(entry.semantics ?? [], written, [entry.library.machineName], false) - } - - /** A deterministic sub-content id: generated packages are the same bytes every run. */ - nextId() { - this.ids += 1 - return `00000000-0000-4000-8000-${this.ids.toString(16).padStart(12, '0')}` - } - - /** - * @param {any[]} fields - * @param {any} written the hand-written object for these fields, or undefined - * @param {string[]} stack the content types this value sits inside, to keep a type out of itself - * @param {boolean} lenient inside an optional group nobody wrote: defaults only, as the editor - * saves a group left closed — and so no sample image for a field required only within it - */ - fields(fields, written, stack, lenient) { - const out = {} - const given = isObject(written) ? written : {} - for (const field of fields) { - const value = this.field(field, given[field.name], stack, lenient) - if (value !== undefined) out[field.name] = value - } - // What the semantics do not name is kept as written: the content check removes it, which a - // file may mean to test. - for (const [key, value] of Object.entries(given)) if (!(key in out) && value !== null && !fields.some((field) => field.name === key)) out[key] = value - return out - } - - /** - * @param {any} field - * @param {any} written the hand-written value, `null` to leave the field out, or undefined - * @param {string[]} stack - * @param {boolean} lenient - */ - field(field, written, stack, lenient) { - if (written === null) return undefined - const optional = field.optional || lenient +/** @type {import('@missing-elements/h5p-author').GeneratorOptions} */ +export const sweepOptions = { + lenient: true, + id: (count) => `00000000-0000-4000-8000-${count.toString(16).padStart(12, '0')}`, + fill(field, { generator, stack }) { switch (field.type) { - case 'group': { - const children = field.fields ?? [] - // A group of one field is saved as that field's value, as the editor and the content - // check have it, unless the group is sub-content. - const flattened = children.length === 1 && !field.isSubContent - if (written === undefined && field.default !== undefined) return field.default - if (written !== undefined && !flattened && !isObject(written)) return written - const value = this.fields(children, flattened ? { [children[0].name]: written } : written, stack, optional && written === undefined) - // Saved even when optional and untouched, as the editor saves every group in its form: - // content types read into one without looking (Personality Quiz's `image.file`). - return flattened ? value[children[0].name] : value - } - case 'list': { - const item = { ...field.field, optional: false } - if (Array.isArray(written)) return written.map((each) => this.field(item, each, stack, false)).filter((each) => each !== undefined) - const count = Math.min(Math.max(optional ? 0 : (field.min ?? 0), field.defaultNum ?? (optional ? 0 : 1)), field.max ?? Infinity) - const items = [] - for (let index = 0; index < count; index++) { - const value = this.field(item, undefined, stack, lenient) - if (value !== undefined) items.push(value) - } - return items.length > 0 || !optional ? items : undefined - } case 'library': - if (isObject(written) && typeof written.library === 'string') return this.writtenLibrary(written, stack) - // A default names the content the editor starts with: Interactive Video's summary task. - if (written === undefined && isObject(field.default) && typeof field.default.library === 'string') return this.writtenLibrary(field.default, stack) - return optional ? undefined : this.library(field, stack) + return generator.library(field, stack) case 'text': - if (written !== undefined) return written - if (field.default !== undefined) return field.default - return optional ? undefined : sampleText(field) + return sampleText(field) case 'number': - if (written !== undefined) return written - if (field.default !== undefined) return field.default - return optional ? undefined : Math.max(field.min ?? 0, Math.min(field.max ?? Infinity, 1)) - case 'boolean': - // The editor's checkbox saves false when nobody ticked it. - return written ?? field.default ?? false + return Math.max(field.min ?? 0, Math.min(field.max ?? Infinity, 1)) case 'select': - if (written !== undefined) return written - if (field.default !== undefined) return field.default - return optional || !field.options?.length ? undefined : field.options[0].value + return field.options?.length ? field.options[0].value : undefined case 'image': case 'file': - if (written !== undefined) return written - return optional ? undefined : { ...MEDIA[field.type], copyright: { license: 'U' } } + return { ...MEDIA[field.type], copyright: { license: 'U' } } case 'video': case 'audio': - if (written !== undefined) return written - return optional ? undefined : [{ path: MEDIA[field.type].path, mime: MEDIA[field.type].mime, copyright: { license: 'U' } }] + return [{ path: MEDIA[field.type].path, mime: MEDIA[field.type].mime, copyright: { license: 'U' } }] default: - return written - } - } - - /** - * Sub-content: the first option the pack has and that is not already a parent here, made from - * its own semantics. None at all is left out, which the content check would do anyway. - * @param {any} field - * @param {string[]} stack - */ - library(field, stack) { - if (stack.length > MAX_DEPTH) return undefined - for (const option of field.options ?? []) { - const name = typeof option === 'string' ? option : option?.name - const match = /^(\S+) (\d+)\.(\d+)$/.exec(name ?? '') - if (!match || stack.includes(match[1])) continue - const entry = this.find(match[1], Number(match[2])) - if (!entry) continue - return this.subContent(entry, undefined, stack) - } - return undefined - } - - /** - * Sub-content written by hand: the library it names, its parameters filled in from that - * library's semantics. A library the pack does not have is kept as written, for the check to - * remove or the runtime to refuse. - * @param {any} written - * @param {string[]} stack - */ - writtenLibrary(written, stack) { - const match = /^(\S+) (\d+)\.(\d+)$/.exec(written.library) - const entry = match ? this.find(match[1], Number(match[2])) : null - if (!entry) return written - return { ...this.subContent(entry, written.params, stack), ...omit(written, ['params']), library: libraryString(entry.library) } - } - - /** - * @param {PackLibrary} entry - * @param {any} written - * @param {string[]} stack - */ - subContent(entry, written, stack) { - return { - library: libraryString(entry.library), - params: this.fields(entry.semantics ?? [], written, [...stack, entry.library.machineName], false), - subContentId: this.nextId(), - metadata: { contentType: entry.library.title, license: 'U', title: entry.library.title } + return undefined } } } @@ -212,21 +61,6 @@ export function mediaIn(params) { return /** @type {(keyof typeof MEDIA)[]} */ (Object.keys(MEDIA)).filter((kind) => text.includes(`"${MEDIA[kind].path}"`)) } -/** Every `library` string in the parameters: the sub-content a package plays. */ -export function librariesIn(params, found = new Set()) { - if (Array.isArray(params)) for (const item of params) librariesIn(item, found) - else if (isObject(params)) { - if (typeof params.library === 'string' && /^\S+ \d+\.\d+$/.test(params.library)) found.add(params.library) - for (const value of Object.values(params)) librariesIn(value, found) - } - return found -} - -/** `H5P.MultiChoice 1.16`, the form `library` fields and `h5p.json` use. */ -export function libraryString(library) { - return `${library.machineName} ${library.majorVersion}.${library.minorVersion}` -} - /** * Text for a required field: its label, as an editor would show it, as markup for an HTML * field, or the first sample its pattern accepts. @@ -245,11 +79,3 @@ function sampleText(field) { function escapeHtml(text) { return text.replace(/[&<>"]/g, (character) => ({ '&': '&', '<': '<', '>': '>', '"': '"' })[character]) } - -function omit(object, keys) { - return Object.fromEntries(Object.entries(object).filter(([key]) => !keys.includes(key))) -} - -function isObject(value) { - return value !== null && typeof value === 'object' && !Array.isArray(value) -} diff --git a/apps/demo/scripts/sweep-content-types.mjs b/apps/demo/scripts/sweep-content-types.mjs index 7ce309bb..78e84313 100644 --- a/apps/demo/scripts/sweep-content-types.mjs +++ b/apps/demo/scripts/sweep-content-types.mjs @@ -6,18 +6,19 @@ import { extname, join, normalize, resolve } from 'node:path' import { crc32, deflateSync } from 'node:zlib' import { BlobReader, BlobWriter, TextReader, Uint8ArrayReader, Uint8ArrayWriter, ZipReader, ZipWriter, configure } from '@zip.js/zip.js' import { chromium } from 'playwright' -import { ContentGenerator, MEDIA, libraryString, librariesIn, mediaIn } from './lib/sweep-content.mjs' +import { ContentGenerator, MEDIA, libraryString, librariesIn, mediaIn, sweepOptions } from './lib/sweep-content.mjs' /** * Plays every content type in the library pack and reports what each one does: whether it boots, * played `untrusted` and trusted, and what it sends when worked through. Built on 2026-10-14; the * hub serves no example content, so each package is made here. * - * Each content type's content is generated from its own `semantics.json` (`lib/sweep-content.mjs`), - * and `sweep/.json`, where one exists, adds what generation cannot give — a drop - * zone, words that cross — and the element's attributes for it (`attributes`). The packages are - * content-only, as h5p.com and h5p.org export, so the pack supplies every library: this is the - * pack's own test as much as the player's. Run it with the player built (`pnpm build`): + * Each content type's content is generated from its own `semantics.json` by the generator of + * @missing-elements/h5p-author, run as `lib/sweep-content.mjs` says, and `sweep/.json`, + * where one exists, adds what generation cannot give — a drop zone, words that cross — and the + * element's attributes for it (`attributes`). The packages are content-only, as h5p.com and + * h5p.org export, so the pack supplies every library: this is the pack's own test as much as the + * player's. Run it with the player built (`pnpm build`): * * pnpm sweep every content type, both modes * pnpm sweep --only H5P.Blanks one or more, comma-separated @@ -52,7 +53,7 @@ const SETTLE = 1_500 for (const mode of modes) if (mode !== 'untrusted' && mode !== 'trusted') fail(`--modes: unknown mode "${mode}"`) const pack = await readPack(packPath) -const generator = new ContentGenerator(pack) +const generator = new ContentGenerator(pack, sweepOptions) if (args.semantics) { const entry = generator.find(args.semantics) if (!entry) fail(`--semantics: no ${args.semantics} in the pack`) diff --git a/apps/demo/tests/sweep-content.test.ts b/apps/demo/tests/sweep-content.test.ts index c2d0a37b..764494ef 100644 --- a/apps/demo/tests/sweep-content.test.ts +++ b/apps/demo/tests/sweep-content.test.ts @@ -1,5 +1,10 @@ import { describe, expect, it } from 'vitest' -import { ContentGenerator, MEDIA, librariesIn, mediaIn } from '../scripts/lib/sweep-content.mjs' +import { ContentGenerator, MEDIA, librariesIn, mediaIn, sweepOptions } from '../scripts/lib/sweep-content.mjs' + +/** + * The sweep's way of running @missing-elements/h5p-author's generator: what it fills a required + * field with so that every content type plays. The generator's own rules are tested in the package. + */ /** A pack of the libraries a test names: folder -> library.json and semantics.json. */ function pack(libraries: { machineName: string; major?: number; minor?: number; title?: string; semantics: unknown[] }[]) { @@ -14,12 +19,11 @@ function pack(libraries: { machineName: string; major?: number; minor?: number; const TEXT = { machineName: 'H5P.AdvancedText', minor: 1, title: 'Text', semantics: [{ name: 'text', type: 'text', widget: 'html', label: 'Text' }] } function generate(semantics: unknown[], written?: unknown, others: Parameters[0] = []): Record { - const libraries = pack([{ machineName: 'H5P.Test', semantics }, ...others]) - const generator = new ContentGenerator(libraries) + const generator = new ContentGenerator(pack([{ machineName: 'H5P.Test', semantics }, ...others]), sweepOptions) return generator.generate(generator.find('H5P.Test')!, written) } -describe('content generated from semantics', () => { +describe('content the sweep generates', () => { it('takes every default, and the smallest valid value for a required field without one', () => { const params = generate([ { name: 'title', type: 'text', default: 'Hello' }, @@ -53,42 +57,6 @@ describe('content generated from semantics', () => { expect(params).toEqual({ color: '#808080', date: '2026/1/1' }) }) - it('saves a group of one field as that field, and a group default as the whole value', () => { - const params = generate([ - { name: 'single', type: 'group', fields: [{ name: 'only', type: 'text', default: 'x' }] }, - { name: 'sub', type: 'group', isSubContent: true, fields: [{ name: 'only', type: 'text', default: 'y' }] }, - { name: 'size', type: 'group', default: { width: 620, height: 310 }, fields: [{ name: 'width', type: 'number' }, { name: 'height', type: 'number' }] } - ]) - expect(params).toEqual({ single: 'x', sub: { only: 'y' }, size: { width: 620, height: 310 } }) - }) - - it('fills an optional group nobody wrote with defaults only, as the editor saves one left closed', () => { - const params = generate([ - { - name: 'image', - type: 'group', - optional: true, - fields: [ - { name: 'file', type: 'image' }, - { name: 'alt', type: 'text' }, - { name: 'decorative', type: 'boolean' }, - { name: 'position', type: 'select', default: 'left', options: [{ value: 'left' }] } - ] - } - ]) - expect(params).toEqual({ image: { decorative: false, position: 'left' } }) - }) - - it('makes a list as long as its minimum or default number, and leaves an optional one out', () => { - const params = generate([ - { name: 'answers', type: 'list', min: 2, field: { name: 'answer', type: 'text', default: 'a' } }, - { name: 'cards', type: 'list', defaultNum: 3, field: { name: 'card', type: 'text', default: 'c' } }, - { name: 'one', type: 'list', field: { name: 'item', type: 'text', default: 'i' } }, - { name: 'keywords', type: 'list', optional: true, field: { name: 'keyword', type: 'text' } } - ]) - expect(params).toEqual({ answers: ['a', 'a'], cards: ['c', 'c', 'c'], one: ['i'] }) - }) - it('fills a library field with the first option the pack has, never the content type itself', () => { const params = generate( [{ name: 'content', type: 'library', options: ['H5P.Test 1.0', 'H5P.Missing 1.0', 'H5P.AdvancedText 1.1'] }], @@ -105,54 +73,8 @@ describe('content generated from semantics', () => { }) }) - it('uses a library field default as written content', () => { - const params = generate([{ name: 'task', type: 'library', default: { library: 'H5P.AdvancedText 1.0', params: { text: '

Given

' } }, options: ['H5P.AdvancedText 1.1'] }], undefined, [TEXT]) - expect(params.task).toMatchObject({ library: 'H5P.AdvancedText 1.1', params: { text: '

Given

' } }) - }) -}) - -describe('hand-written content laid in', () => { - const QUESTION = [ - { name: 'question', type: 'text', widget: 'html', label: 'Question' }, - { - name: 'answers', - type: 'list', - min: 1, - field: { name: 'answer', type: 'group', fields: [{ name: 'text', type: 'text', widget: 'html', label: 'Text' }, { name: 'correct', type: 'boolean' }, { name: 'tip', type: 'text', default: '' }] } - }, - { name: 'behaviour', type: 'group', fields: [{ name: 'random', type: 'boolean', default: true }, { name: 'retry', type: 'boolean', default: true }] } - ] - - it('keeps what is written, fills in the rest, and keeps a written list as long as it is', () => { - const params = generate(QUESTION, { - answers: [{ text: '
Right
', correct: true }, { text: '
Wrong
' }], - behaviour: { random: false } - }) - expect(params).toEqual({ - question: '

Question

', - answers: [ - { text: '
Right
', correct: true, tip: '' }, - { text: '
Wrong
', correct: false, tip: '' } - ], - behaviour: { random: false, retry: true } - }) - }) - - it('removes a field written as null, and keeps a key the semantics do not name', () => { - const params = generate(QUESTION, { question: null, unknown: 1 }) - expect(params).not.toHaveProperty('question') - expect(params.unknown).toBe(1) - }) - - it('fills written sub-content from its own semantics, at the version the pack has', () => { - const params = generate([{ name: 'content', type: 'library', options: ['H5P.AdvancedText 1.1'] }], { content: { library: 'H5P.AdvancedText 1.0', params: {} } }, [TEXT]) - expect(params.content).toMatchObject({ library: 'H5P.AdvancedText 1.1', params: { text: '

Text

' } }) - expect(params.content.subContentId).toMatch(/^00000000-0000-4000-8000-/) - }) - - it('keeps sub-content of a library the pack does not have as written', () => { - const written = { library: 'H5P.Elsewhere 1.0', params: { a: 1 } } - expect(generate([{ name: 'content', type: 'library', options: [] }], { content: written }).content).toEqual(written) + it('keeps a key the semantics do not name, which the content check is then to remove', () => { + expect(generate([{ name: 'title', type: 'text', default: 'x' }], { unknown: 1 }).unknown).toBe(1) }) }) diff --git a/apps/skills/package.json b/apps/skills/package.json index 29e0fbc5..5799ac95 100644 --- a/apps/skills/package.json +++ b/apps/skills/package.json @@ -7,6 +7,7 @@ "typecheck": "tsc -p tsconfig.json" }, "devDependencies": { + "@missing-elements/h5p-author": "workspace:*", "@missing-elements/h5p-libraries": "workspace:*", "@missing-elements/h5p-verify": "workspace:*", "@types/node": "^26.6.5", diff --git a/apps/skills/tests/author-playback.test.ts b/apps/skills/tests/author-playback.test.ts index 9a162e61..b0cd23bf 100644 --- a/apps/skills/tests/author-playback.test.ts +++ b/apps/skills/tests/author-playback.test.ts @@ -4,11 +4,16 @@ import { join, resolve } from 'node:path' import { chromium, type Browser, type Page } from 'playwright' import { afterAll, beforeAll, expect, it } from 'vitest' import { previewPackage } from '../../../packages/verify/lib/preview.mjs' -import { buildQuiz, type Quiz } from '../../../skills/h5p-author/scripts/build-quiz.mjs' +import { buildPackage } from '@missing-elements/h5p-author' const project = resolve(import.meta.dirname, '..') // The pack this project installed, named to the preview as `--libraries` names a copy of its parts. const parts = resolve(project, 'node_modules/@missing-elements/h5p-libraries/parts') +/** The skill's quiz format, as the scenario example writes it. */ +type Quiz = { + kind: string, passPercentage?: number, passedFeedback?: string, failedFeedback?: string, + questions: { title?: string, prompt: string, multipleAnswers?: boolean, answers: { text: string, correct: boolean, feedback?: string }[] }[] +} const source: Quiz = JSON.parse(await readFile(resolve(project, '../../skills/h5p-author/assets/phishing-scenarios.json'), 'utf8')) const example: Quiz = JSON.parse(await readFile(resolve(project, '../../skills/h5p-author/assets/quiz.json'), 'utf8')) const single: Quiz = { ...example, kind: 'multiple-choice', questions: [example.questions[0]] } @@ -20,9 +25,9 @@ let browser: Browser beforeAll(async () => { dir = await mkdtemp(join(tmpdir(), 'h5p-author-interaction-')) file = join(dir, 'scenarios.h5p') - await buildQuiz(source, file, project) + await buildPackage(source, file) singleFile = join(dir, 'single.h5p') - await buildQuiz(single, singleFile, project) + await buildPackage(single, singleFile) browser = await chromium.launch() }) afterAll(async () => { diff --git a/apps/skills/tests/author-verify.test.ts b/apps/skills/tests/author-verify.test.ts index b3ebdcbb..034b9db1 100644 --- a/apps/skills/tests/author-verify.test.ts +++ b/apps/skills/tests/author-verify.test.ts @@ -1,69 +1,142 @@ -import { mkdtemp, readFile, rm } from 'node:fs/promises' +import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join, resolve } from 'node:path' +import { crc32, deflateSync } from 'node:zlib' +import { BlobReader, Uint8ArrayWriter, ZipReader } from '@zip.js/zip.js' +import { buildPackage, schemaOf } from '@missing-elements/h5p-author' import { verifyPackage } from '@missing-elements/h5p-verify' -import { afterAll, beforeAll, expect, it } from 'vitest' -import { buildQuiz, type Quiz } from '../../../skills/h5p-author/scripts/build-quiz.mjs' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' const project = resolve(import.meta.dirname, '..') // The pack this project installed, named to the check as `--libraries` names a copy of its parts. const parts = resolve(project, 'node_modules/@missing-elements/h5p-libraries/parts') const skill = resolve(project, '../../skills/h5p-author') -const example: Quiz = JSON.parse(await readFile(join(skill, 'assets/quiz.json'), 'utf8')) -const scenarios: Quiz = JSON.parse(await readFile(join(skill, 'assets/phishing-scenarios.json'), 'utf8')) +const types = join(skill, 'references/types') +const example = JSON.parse(await readFile(join(skill, 'assets/quiz.json'), 'utf8')) +const scenarios = JSON.parse(await readFile(join(skill, 'assets/phishing-scenarios.json'), 'utf8')) +const META = { title: 'Example', language: 'en', license: 'CC BY', authors: [{ name: 'Ann Author', role: 'Author' }] } +const files = (await readdir(types)).filter((name) => name.endsWith('.md')).sort() let dir: string -const files = new Map() +let clip: Uint8Array + +/** The source a type file gives as its example, the metadata every source carries added. */ +async function exampleOf(file: string) { + const text = await readFile(join(types, file), 'utf8') + const block = /## Example\s+```json\n([\s\S]*?)\n```/.exec(text) + if (!block) throw new Error(`${file} has no example`) + return { ...META, ...JSON.parse(block[1]) } +} + +/** Every file an example names, by its `path`. */ +function paths(value: unknown, found: string[] = []): string[] { + if (Array.isArray(value)) value.forEach((item) => paths(item, found)) + else if (value && typeof value === 'object') { + const object = value as Record + if (typeof object.path === 'string') found.push(object.path) + Object.values(object).forEach((item) => paths(item, found)) + } + return found +} + +/** A PNG of one colour. */ +function png(width: number, height: number) { + const chunk = (type: string, data: Buffer) => { + const head = Buffer.alloc(8) + head.writeUInt32BE(data.length) + head.write(type, 4) + const crc = Buffer.alloc(4) + crc.writeUInt32BE(crc32(Buffer.concat([Buffer.from(type), data]))) + return Buffer.concat([head, data, crc]) + } + const header = Buffer.alloc(13) + header.writeUInt32BE(width, 0) + header.writeUInt32BE(height, 4) + header[8] = 8 + header[9] = 2 + const row = Buffer.alloc(1 + width * 3, 0x90) + row[0] = 0 + return Buffer.concat([Buffer.from([137, 80, 78, 71, 13, 10, 26, 10]), chunk('IHDR', header), chunk('IDAT', deflateSync(Buffer.concat(Array.from({ length: height }, () => row)))), chunk('IEND', Buffer.alloc(0))]) +} + +/** A second of silence as MPEG-1 Layer III, 128 kbit/s at 44.1 kHz. */ +function mp3() { + const frame = Buffer.alloc(417) + frame.set([0xff, 0xfb, 0x90, 0x64]) + return Buffer.concat(Array.from({ length: 38 }, () => frame)) +} + +/** A sample file for each name an example gives, by its type: what the user would supply. */ +function sample(name: string) { + const extension = name.split('.').pop() + if (extension === 'png') return png(480, 270) + if (extension === 'mp4') return clip + if (extension === 'mp3') return mp3() + if (extension === 'vtt') return Buffer.from('WEBVTT\n\n00:00.000 --> 00:02.000\nA river.\n') + throw new Error(`No sample for ${name}`) +} beforeAll(async () => { dir = await mkdtemp(join(tmpdir(), 'h5p-author-verify-')) - for (const kind of ['multiple-choice', 'question-set'] as const) { - const source: Quiz = { ...example, kind, questions: kind === 'multiple-choice' ? [example.questions[0]] : example.questions } - const file = join(dir, `${kind}.h5p`) - await buildQuiz(source, file, project) - files.set(kind, file) + // The demo's clip, which the player's own tests play: an H.264 mp4 of ten seconds. + const reader = new ZipReader(new BlobReader(new Blob([await readFile(resolve(project, '../demo/demo/content/video.h5p'))]))) + try { + const entry = (await reader.getEntries()).find((each) => each.filename.endsWith('.mp4'))! + clip = await (entry as import('@zip.js/zip.js').FileEntry).getData(new Uint8ArrayWriter()) + } finally { + await reader.close() } - const file = join(dir, 'scenarios.h5p') - await buildQuiz(scenarios, file, project) - files.set('scenarios', file) - // Interface texts from the libraries' Ukrainian, one of them with an apostrophe the content check escapes. - const translated = join(dir, 'ukrainian.h5p') - await buildQuiz({ ...example, language: 'uk' }, translated, project) - files.set('ukrainian', translated) - // Titles and an author's name with quotes and markup, which the content check escapes. - const punctuation = join(dir, 'punctuation.h5p') - await buildQuiz({ - ...example, - authors: [{ name: "Anne O'Brien ", role: 'Author' }], - questions: example.questions.map(question => ({ ...question, title: `${question.title}'s "draft" & co` })) - }, punctuation, project) - files.set('punctuation', punctuation) }) afterAll(async () => { if (dir) await rm(dir, { recursive: true, force: true }) }) -// Step 4 of the skill: the check, through the verifier's published API. -it.each([ - ['multiple-choice', false], - ['multiple-choice', true], - ['question-set', false], - ['question-set', true], - ['scenarios', false], - ['scenarios', true], - ['ukrainian', false], - ['ukrainian', true], - ['punctuation', false], - ['punctuation', true] -])('plays authored %s (untrusted: %s) without warnings', async (kind, untrusted) => { - const report = await verifyPackage({ - file: files.get(kind)!, - untrusted, - libraries: untrusted ? parts : undefined, - settle: 500 +/** Step 4 of the skill: the check, through the verifier's published API, trusted and untrusted. */ +async function playsBothWays(file: string) { + for (const untrusted of [false, true]) { + const report = await verifyPackage({ file, untrusted, libraries: untrusted ? parts : undefined, settle: 500 }) + expect(report.verdict, JSON.stringify(report)).toBe('pass') + expect(report.warnings, `${file}, untrusted: ${untrusted}`).toEqual([]) + expect(report.errors).toEqual([]) + expect(report.state).toBe('ready') + expect(report.untrusted).toBe(untrusted) + expect(report.revision).toMatch(/^sha256:[0-9a-f]{64}(; libraries sha256:[0-9a-f]{64})?$/) + } +} + +describe.concurrent('every type file', () => { + it.each(files)('%s: its example builds, and plays trusted and untrusted without warnings', async (file) => { + const source = await exampleOf(file) + const folder = join(dir, file.slice(0, -'.md'.length)) + await mkdir(folder, { recursive: true }) + for (const path of paths(source.params)) if (!/^https?:/.test(path)) await writeFile(join(folder, path), sample(path)) + const output = join(folder, 'example.h5p') + const result = await buildPackage(source, output, { base: folder }) + expect(result.notes).toEqual([]) + await playsBothWays(output) + }, 180_000) + + it.each(files)('%s: its fields are the pack\'s, as `h5p-author schema --markdown` writes them', async (file) => { + const text = await readFile(join(types, file), 'utf8') + // Out of date after the pack changed: run `pnpm --filter @missing-elements/h5p-author sync:skill`. + expect(text).toContain(await schemaOf(file.slice(0, -'.md'.length), { markdown: true })) }) - expect(report.verdict, JSON.stringify(report)).toBe('pass') - expect(report.warnings).toEqual([]) - expect(report.errors).toEqual([]) - expect(report.state).toBe('ready') - expect(report.untrusted).toBe(untrusted) - expect(report.revision).toMatch(/^sha256:[0-9a-f]{64}(; libraries sha256:[0-9a-f]{64})?$/) - expect(report.rendered?.text).toBeGreaterThan(100) -}, 60_000) +}) + +describe.concurrent('the quiz format', () => { + const quizzes: [string, unknown][] = [ + ['multiple choice', { ...example, kind: 'multiple-choice', questions: [example.questions[0]] }], + ['question set', example], + ['eight scenarios', scenarios], + // Interface texts from the libraries' Ukrainian, one of them with an apostrophe the content check escapes. + ['Ukrainian', { ...example, language: 'uk' }], + // Titles and an author's name with quotes and markup, which the content check escapes. + ['punctuation', { + ...example, + authors: [{ name: "Anne O'Brien ", role: 'Author' }], + questions: example.questions.map((question: { title: string }) => ({ ...question, title: `${question.title}'s "draft" & co` })) + }] + ] + it.each(quizzes)('%s plays trusted and untrusted without warnings', async (name, source) => { + const output = join(dir, `${name.replaceAll(' ', '-')}.h5p`) + await buildPackage(source, output) + await playsBothWays(output) + }, 180_000) +}) diff --git a/docs/development.md b/docs/development.md index af4f38f2..acbbf094 100644 --- a/docs/development.md +++ b/docs/development.md @@ -1,9 +1,10 @@ # Development A pnpm workspace: `packages/player`, `packages/runtime`, `packages/libraries`, -`packages/normalize`, `packages/verify`, `packages/cmi5`, `packages/embed`, the demo site in -`apps/demo`, and in `apps/skills` the tests of the agent skills' scripts, run from a project the -way a skill's user runs them. +`packages/normalize`, `packages/verify`, `packages/author`, `packages/cmi5`, `packages/embed`, +`packages/vscode`, the demo site in `apps/demo`, and in `apps/skills` the tests of the agent +skills, run from a project the way a skill's user runs them: every h5p-author type file's example +built and played. ```bash pnpm install diff --git a/docs/verify.md b/docs/verify.md index 68199e92..15eeebb7 100644 --- a/docs/verify.md +++ b/docs/verify.md @@ -36,10 +36,11 @@ library pack, so an export without its libraries plays offline. For authoring, the [`h5p-author` skill](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) -adds a prompt-to-quiz workflow and a portable packaging helper for text-only Multiple Choice -and Question Set. It takes a small quiz JSON file, fills the installed libraries' defaults, -includes their runtime dependencies, then requires this playback check and screenshot review, -and opens the quiz in the preview for a person to answer every question. The source JSON remains -editable; the `.h5p` and full revision from the verification report identify what was checked. -Answer accuracy and a complete question-by-question interaction check remain separate from the +adds a prompt-to-package workflow for 28 content types, on +[`@missing-elements/h5p-author`](https://github.com/missing-elements/h5p-offline-player/tree/main/packages/author#readme). +It takes a short JSON source of only what differs from a type's defaults, fills the rest from the +libraries' semantics and translations, includes their runtime dependencies, then requires this +playback check and screenshot review, and opens the package in the preview for a person to work +through. The source JSON remains editable; the `.h5p` and full revision from the verification +report identify what was checked. Whether the content is right remains separate from the verifier's startup verdict. diff --git a/package.json b/package.json index fb6fa9ea..094b1987 100644 --- a/package.json +++ b/package.json @@ -15,6 +15,7 @@ "typecheck": "pnpm -r typecheck", "normalize": "node packages/normalize/bin/h5p-normalize.mjs", "verify": "node packages/verify/bin/h5p-verify.mjs", + "author": "node packages/author/bin/h5p-author.mjs", "demo:content": "pnpm --filter @missing-elements/h5p-demo demo:content", "demo:og": "pnpm --filter @missing-elements/h5p-demo demo:og", "demo:icons": "pnpm --filter @missing-elements/h5p-demo demo:icons", diff --git a/packages/author/README.md b/packages/author/README.md new file mode 100644 index 00000000..164480ec --- /dev/null +++ b/packages/author/README.md @@ -0,0 +1,127 @@ +# @missing-elements/h5p-author + +Builds a playable `.h5p` of any content type in the H5P hub from a short JSON source: the content +type, and only the fields that differ from its defaults. The rest comes from the libraries' own +`semantics.json` and their translations for the content's language; the libraries come from +[@missing-elements/h5p-libraries](https://www.npmjs.com/package/@missing-elements/h5p-libraries), +the pack this release pins, each checked against the pack's SHA-256 index. No H5P server, no +editor, no network but for media named by web address. Needs Node 22 or later; write `@latest`, +since without it npm on an older Node quietly installs an older release. + +```bash +npx @missing-elements/h5p-author@latest build rivers.json rivers.h5p +``` + +```json +{ + "type": "H5P.Blanks", + "title": "Rivers of Europe", + "language": "de", + "license": "CC BY", + "authors": [{ "name": "Ann Author", "role": "Author" }], + "params": { + "questions": ["Die *Donau* fließt ins Schwarze Meer.", "Paris liegt an der *Seine*."] + } +} +``` + +``` +Built /work/rivers.h5p: H5P.Blanks, 9 libraries from the library pack 1.0.0. +H5P.Blanks 1.14: interface texts from its translation language/de.json. +Not verified: run npx @missing-elements/h5p-verify@latest on it and look at its screenshot before handing it over. +``` + +It is written for an AI agent and its tokens: the agent writes what only an author can know, and +code writes everything that can be derived. The +[h5p-author skill](https://github.com/missing-elements/h5p-offline-player/tree/main/skills/h5p-author) +is the workflow around it: choosing the content type with the user, a file per type with notes, +an example and the fields, then the check and the preview. + +## The source + +| Field | Meaning | +|---|---| +| `type` | The content type's machine name, such as `H5P.Blanks`; the pack's newest version is used | +| `title` | The title, at most 255 characters | +| `language` | A BCP 47 tag, `en`, `de-AT`; it picks the libraries' translation of their interface texts | +| `license` | An H5P licence code: `U` (undisclosed), `CC BY`, `CC BY-SA`, `CC BY-ND`, `CC BY-NC`, `CC BY-NC-SA`, `CC BY-NC-ND`, `CC0 1.0`, `CC PDM`, `GNU GPL`, `PD`, `ODC PDDL`, `C`; `licenseVersion` for a CC BY one | +| `authors` | `[{ "name": "…", "role": "Author" }]`, the role one of Author, Editor, Licensee, Originator | +| `params` | The content type's fields that differ from their defaults | + +What `params` takes is what H5P's editor would let an author save in the same form: + +- **Text**: plain text in a plain field, stored escaped as the platforms store it; in a field with + tags, markup with those tags only, or plain text, which is wrapped in paragraphs as the editor + wraps typing. A tag the field does not take, an event handler or a script address is refused. +- **Choices, numbers, lists**: a select's options, a number's range, a list's size, all checked. + A field the semantics do not name is refused, so a misspelt one is never dropped in silence. +- **Sub-content**: `{ "library": "H5P.MultiChoice", "params": { … }, "metadata": { "title": "…" } }`, + filled in the same way, at the pack's version. +- **Files**: `{ "path": "photo.jpg" }`, a file beside the source or an `http(s)` address, with an + optional `copyright`. Each is copied into the package with its type and, for a picture, its size; + the package is then laid out to stream, media stored and an mp4's index first. A YouTube, Vimeo + or Panopto address stays where it is. +- **Interface texts** (Check, Retry…): leave them out; they come from the libraries' translations, + and the command lists what stayed English. A source may still set one by name. + +Some content types have a widget of their own in the editor, which sets fields the form hides. +The build does what those widgets do: it lays out a Course Presentation slide's elements, places +Interactive Video interactions, joins Game Map stages and sets their ids, numbers Branching +Scenario end screens, and names untitled Question Set questions. It refuses what they would not +let pass: a scenario path with no end, a map with no start, a hotspot question with no right area. + +A source with problems writes nothing and lists every problem by its path at once: + +``` +h5p-author: The source has 2 problems: + params.correct: "yes" is not an option; the options are "true", "false" + params.behaviour.sideways: is not a field here; the fields are enableRetry, enableSolutionsButton, … +``` + +The quiz format the h5p-author skill published first (`{ "kind": "question-set", "questions": [ … ] }`) +is still taken, for a Multiple Choice question or a Question Set of them. + +## A content type's fields + +```bash +npx @missing-elements/h5p-author@latest schema H5P.TrueFalse +``` + +``` +H5P.TrueFalse 1.8: True/False Question +- `media` group — Media + - `type` sub-content: H5P.Image | H5P.Video | H5P.Audio — Type: Optional media to display above the question. +- `question` html: strong em sub sup h2 h3 pre code, required — Question +- `correct` one of "true" | "false" = "true" — Correct answer +… +``` + +`--all` adds the interface texts; `--markdown` prints the section the skill's type files carry. +An unknown name lists the pack's content types. + +## Options + +- `--libraries `: another copy of the pack's `parts/` folder, in place of the pinned one. + +## API + +```js +import { buildPackage, schemaOf, SourceError } from '@missing-elements/h5p-author' + +const { output, manifest, content, libraries, translations, notes, media } = + await buildPackage(source, 'rivers.h5p', { base: process.cwd() }) +``` + +`buildPackage(source, output, { libraries, base, fetch })` refuses an existing output, and throws a +`SourceError` whose `problems` are `{ path, message }`. `schemaOf(machineName, { markdown, all, +libraries })` returns the field list. `ContentGenerator` is the engine underneath, also run by the +repository's sweep of every content type. + +## What it does not do + +It checks structure, not pedagogy: whether the answers are right, the questions fair, or the +pictures placed well is for a person to judge, in +[h5p-verify](https://www.npmjs.com/package/@missing-elements/h5p-verify)'s `--preview`. It does not +edit an existing package. The licence of each library is in the pack's `libraries.txt`. + +MIT. diff --git a/packages/author/bin/h5p-author.mjs b/packages/author/bin/h5p-author.mjs new file mode 100755 index 00000000..3083129f --- /dev/null +++ b/packages/author/bin/h5p-author.mjs @@ -0,0 +1,89 @@ +#!/usr/bin/env node +import { readFile } from 'node:fs/promises' +import { dirname, resolve } from 'node:path' +import { buildPackage, schemaOf, SourceError } from '../lib/author.mjs' +import { packVersion } from '../lib/pack.mjs' + +const USAGE = `Usage: + h5p-author build [--libraries ] + h5p-author schema [--markdown] [--all] [--libraries ] + +build Makes a playable .h5p from a source: { "type", "title", "language", "license", + "authors", "params" }, or a quiz in the h5p-author skill's quiz format. Writes only + when the whole source is right, and never over an existing file. +schema Prints a content type's fields as a source writes them: --markdown as a skill's + type file carries them, --all with the interface texts. + +The libraries come from the pack this release pins (@missing-elements/h5p-libraries); +--libraries names another copy's parts/ folder. Check every package with +npx @missing-elements/h5p-verify@latest before handing it over.` + +/** @param {string[]} argv */ +function parse(argv) { + /** @type {string[]} */ + const positional = [] + /** @type {Record} */ + const flags = {} + for (let i = 0; i < argv.length; i++) { + const arg = argv[i] + if (arg === '--libraries') { + const value = argv[++i] + if (!value || value.startsWith('--')) throw new Error('--libraries needs a folder') + flags.libraries = value + } else if (arg.startsWith('--')) flags[arg.slice(2)] = true + else positional.push(arg) + } + return { positional, flags } +} + +try { + const { positional, flags } = parse(process.argv.slice(2)) + const [command, ...rest] = positional + if (flags.version) { + const own = JSON.parse(await readFile(new URL('../package.json', import.meta.url), 'utf8')).version + console.log(`@missing-elements/h5p-author ${own}, library pack ${await packVersion()}`) + } else if (flags.help || !command) { + console.log(USAGE) + if (!command && !flags.help) process.exitCode = 1 + } else if (command === 'schema') { + if (rest.length !== 1) throw new Error('schema takes one content type, such as H5P.Blanks') + for (const flag of Object.keys(flags)) if (!['markdown', 'all', 'libraries'].includes(flag)) throw new Error(`schema does not take --${flag}`) + console.log(await schemaOf(rest[0], { markdown: Boolean(flags.markdown), all: Boolean(flags.all), libraries: /** @type {string | undefined} */ (flags.libraries) })) + } else if (command === 'build') { + if (rest.length !== 2) throw new Error('build takes a source and an output: h5p-author build quiz.json quiz.h5p') + for (const flag of Object.keys(flags)) if (flag !== 'libraries') throw new Error(`build does not take --${flag}`) + const sourcePath = resolve(rest[0]) + let source + try { + source = JSON.parse(await readFile(sourcePath, 'utf8')) + } catch (error) { + throw new Error(`${rest[0]} is not a readable JSON file: ${error instanceof Error ? error.message : error}`) + } + const result = await buildPackage(source, rest[1], { base: dirname(sourcePath), libraries: /** @type {string | undefined} */ (flags.libraries) }) + const from = flags.libraries ? `the pack in ${flags.libraries}` : `the library pack ${await packVersion()}` + console.log(`Built ${result.output}: ${result.manifest.mainLibrary}, ${result.libraries.length} libraries from ${from}${result.media.length ? `, ${result.media.length} media file${result.media.length === 1 ? '' : 's'}` : ''}.`) + const language = String(result.manifest.language) + if (!/^en(?:-|$)/i.test(language)) { + for (const { library, translation, strings, english } of result.translations) { + if (!translation) console.warn(`${library} has no translation for ${language}: its ${strings} interface texts are English.`) + else if (!english.length) console.log(`${library}: interface texts from its translation language/${translation}.json.`) + else { + const list = english.map(({ field, text, dropped }) => ` ${field}: ${JSON.stringify(text)}${dropped ? ` (its translation drops ${dropped.join(' and ')})` : ''}`) + console.warn(`${library}: interface texts from its translation language/${translation}.json; ${english.length} of ${strings} as in English, untranslated or the same word:\n${list.join('\n')}`) + } + } + } + for (const { path, message } of result.notes) console.warn(`Note: ${path} ${message}`) + console.log('Not verified: run npx @missing-elements/h5p-verify@latest on it and look at its screenshot before handing it over.') + } else { + throw new Error(`unknown command "${command}"\n\n${USAGE}`) + } +} catch (error) { + if (error instanceof SourceError) { + console.error(`h5p-author: ${error.message}`) + } else { + console.error(`h5p-author: ${error instanceof Error ? error.message : error}`) + if (error instanceof Error && error.cause instanceof Error) console.error(error.cause.message) + } + process.exitCode = 1 +} diff --git a/packages/author/lib/author.mjs b/packages/author/lib/author.mjs new file mode 100644 index 00000000..c15aabf6 --- /dev/null +++ b/packages/author/lib/author.mjs @@ -0,0 +1,56 @@ +/** + * @missing-elements/h5p-author: a playable .h5p of any content type in the H5P hub's library pack, + * from a short JSON source. The source writes only what the defaults cannot know; everything else + * comes from the libraries' own semantics and translations, and the libraries from the pack this + * package pins, checked against its index. + */ + +import { openPack } from './pack.mjs' +import { schemaLines, schemaMarkdown } from './schema.mjs' +import { LAYOUTS } from './types.mjs' + +export { buildPackage, SourceError } from './build.mjs' +export { ContentGenerator, librariesIn, libraryString } from './generator.mjs' +export { GENERATED_END, GENERATED_START } from './schema.mjs' + +/** + * @typedef {import('./build.mjs').Source} Source + * @typedef {import('./build.mjs').BuildResult} BuildResult + * @typedef {import('./build.mjs').BuildOptions} BuildOptions + * @typedef {import('./generator.mjs').GeneratorOptions} GeneratorOptions + */ + +/** + * A content type's fields as a source writes them, from the pack: one line each. + * @param {string} machineName such as `H5P.Blanks` + * @param {{ libraries?: string, markdown?: boolean, all?: boolean }} [options] + * `libraries`, a parts folder in place of the pinned pack; `markdown`, the section a type file + * carries, between its markers; `all`, the interface texts too + * @returns {Promise} + */ +export async function schemaOf(machineName, options = {}) { + const pack = await openPack(options.libraries) + try { + const entry = pack.newest(machineName) + if (!entry) { + const runnable = [] + for (const each of pack.libraries) { + const part = await pack.open(each) + if (part.library.runnable) runnable.push(`${each.machineName} (${part.library.title})`) + } + throw new Error(`The library pack has no ${machineName}. Its content types: ${[...new Set(runnable)].sort().join(', ')}`) + } + const part = await pack.open(entry) + if (!part.semantics) throw new Error(`${machineName} has no semantics: it is not a content type an author fills in`) + /** @type {(name: string) => boolean} */ + const has = (name) => Boolean(pack.newest(name)) + const sets = LAYOUTS[machineName]?.sets ?? [] + if (options.markdown) return schemaMarkdown(part.library, part.semantics, { has, sets }) + const { lines, interfaceTexts } = schemaLines(part.semantics, { all: options.all, has, sets }) + const header = `${part.library.machineName} ${part.library.majorVersion}.${part.library.minorVersion}: ${part.library.title}` + const footer = !options.all && interfaceTexts ? [`(${interfaceTexts} interface texts left out; --all lists them)`] : [] + return [header, ...lines, ...footer].join('\n') + } finally { + await pack.close() + } +} diff --git a/packages/author/lib/build.mjs b/packages/author/lib/build.mjs new file mode 100644 index 00000000..8fabebb6 --- /dev/null +++ b/packages/author/lib/build.mjs @@ -0,0 +1,383 @@ +import { constants, createReadStream } from 'node:fs' +import { copyFile, mkdtemp, open, rm } from 'node:fs/promises' +import { dirname, join, resolve } from 'node:path' +import { Readable, Writable } from 'node:stream' +import { TextReader, ZipWriter } from '@zip.js/zip.js' +import { normalizeArchive } from '@missing-elements/h5p-normalize' +import { ContentGenerator, librariesIn, libraryString } from './generator.mjs' +import { MediaStore } from './media.mjs' +import { openPack } from './pack.mjs' +import { quizSource, validateQuiz } from './quiz.mjs' +import { escapeText } from './text.mjs' +import { localize, translationFiles } from './translate.mjs' +import { LAYOUTS, patchSemantics, placesOf } from './types.mjs' + +/** + * A source made a package: validated, filled in from the pack's semantics and translations, + * laid out, its media copied and described, and written with the libraries it needs, each part + * checked against the pack's index. Nothing is written unless all of it is right: every problem + * the source has is listed at once, each by its path, so a fix takes one round. + */ + +/** The licences H5P's metadata takes. */ +const LICENSES = ['U', 'CC BY', 'CC BY-SA', 'CC BY-ND', 'CC BY-NC', 'CC BY-NC-SA', 'CC BY-NC-ND', 'CC0 1.0', 'CC PDM', 'GNU GPL', 'PD', 'ODC PDDL', 'C'] +const CC_VERSIONS = ['4.0', '3.0', '2.5', '2.0', '1.0'] +const ROLES = ['Author', 'Editor', 'Licensee', 'Originator'] + +/** + * @typedef {{ name: string, role: string }} Author + * @typedef {{ + * type: string, title: string, language: string, license: string, licenseVersion?: string, + * authors: Author[], source?: string, yearFrom?: number, yearTo?: number, params: Record + * }} Source a package's source: the content type, its metadata, and the parameters that differ from the defaults + * @typedef {{ path: string, message: string }} Problem + * @typedef {import('./translate.mjs').TranslationReport} TranslationReport + * @typedef {{ + * output: string, manifest: Record, content: Record, + * libraries: string[], translations: TranslationReport[], notes: Problem[], media: string[] + * }} BuildResult what was written, and what is worth telling about it + * @typedef {{ libraries?: string, base?: string, fetch?: typeof fetch }} BuildOptions + * `libraries`, a library pack's parts folder in place of the pinned pack; `base`, the folder a + * source's file paths are read from; `fetch`, for media named by web address + */ + +/** A source's problems, all of them, each named by its path. */ +export class SourceError extends Error { + /** @param {Problem[]} problems */ + constructor(problems) { + super(`The source has ${problems.length} problem${problems.length === 1 ? '' : 's'}:\n${problems.map(({ path, message }) => ` ${path}: ${message}`).join('\n')}`) + this.name = 'SourceError' + this.problems = problems + } +} + +/** @param {unknown} value @returns {value is Record} */ +const isObject = (value) => value !== null && typeof value === 'object' && !Array.isArray(value) + +/** + * The general source checked for its own fields; a quiz in the published format mapped onto it. + * @param {unknown} input + * @returns {Source} + * @internal + */ +export function readSource(input) { + if (isObject(input) && 'kind' in input) { + validateQuiz(input) + return /** @type {Source} */ (quizSource(input)) + } + /** @type {Problem[]} */ + const problems = [] + const report = (/** @type {string} */ path, /** @type {string} */ message) => problems.push({ path, message }) + if (!isObject(input)) throw new SourceError([{ path: 'source', message: 'expected an object: { "type", "title", "language", "license", "authors", "params" }' }]) + const known = ['type', 'title', 'language', 'license', 'licenseVersion', 'authors', 'source', 'yearFrom', 'yearTo', 'params'] + for (const key of Object.keys(input)) if (!known.includes(key)) report(key, `is not a field of a source; the fields are ${known.join(', ')}`) + if (typeof input.type !== 'string' || !/^\S+$/.test(input.type.trim())) report('type', 'expected a content type\'s machine name, such as "H5P.Blanks"') + if (typeof input.title !== 'string' || !input.title.trim()) report('title', 'expected the activity\'s title') + else if (input.title.length > 255) report('title', 'takes at most 255 characters') + if (typeof input.language !== 'string') report('language', 'expected a BCP 47 language tag, such as "en" or "de-AT"') + else { + try { + Intl.getCanonicalLocales(input.language) + } catch { + report('language', 'expected a BCP 47 language tag, such as "en" or "de-AT"') + } + } + if (!LICENSES.includes(/** @type {string} */ (input.license))) report('license', `use one of ${LICENSES.map((each) => JSON.stringify(each)).join(', ')}; "U" when it is not known`) + if (input.licenseVersion !== undefined) { + if (!String(input.license).startsWith('CC ') || input.license === 'CC0 1.0' || input.license === 'CC PDM') report('licenseVersion', 'is for a Creative Commons BY licence only') + else if (!CC_VERSIONS.includes(input.licenseVersion)) report('licenseVersion', `use one of ${CC_VERSIONS.join(', ')}`) + } + if (!Array.isArray(input.authors) || !input.authors.length) report('authors', 'name at least one author: [{ "name": "…", "role": "Author" }]') + else input.authors.forEach((/** @type {unknown} */ author, /** @type {number} */ i) => { + if (!isObject(author) || typeof author.name !== 'string' || !author.name.trim()) report(`authors[${i}].name`, 'expected the author\'s name') + else if (!ROLES.includes(author.role)) report(`authors[${i}].role`, `use one of ${ROLES.join(', ')}`) + else for (const key of Object.keys(author)) if (key !== 'name' && key !== 'role') report(`authors[${i}].${key}`, 'is not part of an author; write name and role') + }) + if (input.source !== undefined && (typeof input.source !== 'string' || !/^https?:\/\/\S+$/i.test(input.source))) report('source', 'expected the address the content comes from') + for (const key of ['yearFrom', 'yearTo']) if (input[key] !== undefined && !Number.isInteger(input[key])) report(key, 'expected a year') + if (input.params !== undefined && !isObject(input.params)) report('params', 'expected an object of the content type\'s fields') + if (problems.length) throw new SourceError(problems) + return /** @type {Source} */ ({ ...input, type: String(input.type).trim(), params: input.params ?? {} }) +} + +/** + * The libraries a source can bring in, opened and localized before generation: the content type, + * every library the written parameters name, and every one a library field names as its default. + * @param {import('./pack.mjs').Pack} pack + * @param {Source} source + * @param {(path: string, message: string) => void} report + */ +async function loadLibraries(pack, source, report) { + /** @type {Map} */ + const entries = new Map() + /** @type {TranslationReport[]} */ + const translations = [] + /** @type {Map} */ + const parts = new Map() + const main = /^(\S+?)(?:\s+(\d+)\.\d+)?$/.exec(source.type) + const mainEntry = main ? pack.newest(main[1], main[2] === undefined ? undefined : Number(main[2])) : null + if (!mainEntry) { + report('type', `the library pack has no content type ${source.type}`) + return null + } + /** @type {{ machineName: string, major?: number }[]} */ + const queue = [{ machineName: mainEntry.machineName, major: mainEntry.majorVersion }] + /** @param {unknown} value */ + const named = (value) => { + if (Array.isArray(value)) value.forEach(named) + else if (isObject(value)) { + const match = typeof value.library === 'string' ? /^\s*(\S+?)(?:\s+(\d+)\.\d+)?\s*$/.exec(value.library) : null + // A name without a version is resolved by the field's options, which may name any major the + // pack has: each is loaded. + if (match && match[2] !== undefined) queue.push({ machineName: match[1], major: Number(match[2]) }) + else if (match) for (const entry of pack.libraries) if (entry.machineName === match[1]) queue.push({ machineName: match[1], major: entry.majorVersion }) + Object.values(value).forEach(named) + } + } + named(source.params) + /** @param {any[]} fields */ + const defaults = (fields) => { + for (const field of fields) { + if (field.type === 'library' && isObject(field.default)) named(field.default) + if (field.fields) defaults(field.fields) + if (field.field) defaults([field.field]) + } + } + while (queue.length) { + const { machineName, major } = /** @type {{ machineName: string, major?: number }} */ (queue.shift()) + const entry = pack.newest(machineName, major) + if (!entry || parts.has(entry.folder)) continue + const part = await pack.open(entry) + parts.set(entry.folder, part) + let semantics = part.semantics + if (semantics) { + const files = translationFiles([...part.translations.keys()], source.language).map((name) => ({ name, read: () => pack.translation(part, name) })) + const localized = await localize(semantics, files, libraryString(part.library)) + semantics = patchSemantics(part.library.machineName, localized.semantics) + if (part.library.runnable || entry === mainEntry) translations.push(localized.report) + defaults(semantics) + } + entries.set(entry.folder, { folder: entry.folder, library: part.library, semantics }) + } + // Every library the index knows stays findable, so a name the pre-pass did not see is reported + // as a library of the pack and not as an unknown one; its semantics are read when it is found. + for (const entry of pack.libraries) { + if (!entries.has(entry.folder)) entries.set(entry.folder, { folder: entry.folder, library: { machineName: entry.machineName, majorVersion: entry.majorVersion, minorVersion: entry.minorVersion, title: entry.machineName }, semantics: null }) + } + return { entries, mainEntry, translations, parts } +} + +/** + * The libraries a package carries: what the content names, its dependencies, and the addons the + * pack has whose pattern its text matches, as a site with the addon installed exports it. + * @param {import('./pack.mjs').Pack} pack + * @param {string[]} named library strings, the main library first + * @param {Record} content + */ +async function closure(pack, named, content) { + /** @type {Map} */ + const selected = new Map() + /** @param {string} machineName @param {number} [major] @param {number} [minor] */ + const use = async (machineName, major, minor = 0) => { + const entry = pack.newest(machineName, major) + if (!entry || entry.minorVersion < minor) throw new Error(`The library pack cannot supply ${machineName} ${major ?? ''}.${minor}`) + if (selected.has(entry.folder)) return + const part = await pack.open(entry) + selected.set(entry.folder, part) + for (const dependency of [...(part.library.preloadedDependencies ?? []), ...(part.library.dynamicDependencies ?? [])]) { + await use(dependency.machineName, Number(dependency.majorVersion), Number(dependency.minorVersion)) + } + } + for (const name of named) { + const [machineName, version] = name.split(' ') + const [major, minor] = version.split('.').map(Number) + await use(machineName, major, minor) + } + const addons = [] + for (const entry of pack.libraries) { + const patterns = entry.addTo?.content?.types ?? [] + for (const type of patterns) { + const regex = type?.text?.regex + if (typeof regex !== 'string') continue + const match = /^\/(.*)\/([a-z]*)$/s.exec(regex) + if (!match) continue + let pattern + try { + pattern = new RegExp(match[1], match[2].replace(/[^imsuy]/g, '')) + } catch { + continue + } + if (strings(content).some((each) => pattern.test(each))) { + addons.push(entry) + break + } + } + } + for (const addon of addons) await use(addon.machineName, addon.majorVersion, addon.minorVersion) + return { selected, addons: addons.map((addon) => `${addon.machineName} ${addon.majorVersion}.${addon.minorVersion}`) } +} + +/** Every string in a value. @param {unknown} value @param {string[]} [found] */ +function strings(value, found = []) { + if (typeof value === 'string') found.push(value) + else if (Array.isArray(value)) value.forEach((item) => strings(item, found)) + else if (isObject(value)) Object.values(value).forEach((item) => strings(item, found)) + return found +} + +/** + * Builds a package from a source and writes it to `output`, which must not exist yet. + * @param {unknown} input the source: `{ type, title, language, license, authors, params }`, or a quiz in the published quiz format + * @param {string} output + * @param {BuildOptions} [options] + * @returns {Promise} + */ +export async function buildPackage(input, output, options = {}) { + const source = readSource(input) + const pack = await openPack(options.libraries) + output = resolve(output) + let temporary + try { + /** @type {Problem[]} */ + const problems = [] + /** @type {Problem[]} */ + const notes = [] + const report = (/** @type {string} */ path, /** @type {string} */ message) => problems.push({ path, message }) + const note = (/** @type {string} */ path, /** @type {string} */ message) => notes.push({ path, message }) + const loaded = await loadLibraries(pack, source, report) + if (!loaded) throw new SourceError(problems) + const { entries, mainEntry, translations } = loaded + const authors = source.authors.map((author) => ({ name: escapeText(author.name), role: author.role })) + const generator = new ContentGenerator(entries, { + metadata(entry, written, path) { + const title = written?.title + if (title !== undefined && (typeof title !== 'string' || !title.trim())) report(`${path}.metadata.title`, 'expected a title') + else if (typeof title === 'string' && title.length > 255) report(`${path}.metadata.title`, 'takes at most 255 characters') + return { + contentType: escapeText(String(entry.library.title ?? entry.library.machineName)), + license: source.license, + ...(source.licenseVersion ? { licenseVersion: source.licenseVersion } : {}), + authors: authors.map((author) => ({ ...author })), + ...(typeof title === 'string' && title.trim() ? { title: escapeText(title) } : {}) + } + } + }) + const main = /** @type {NonNullable} */ (mainEntry) + const mainLibrary = /** @type {{ folder: string, library: any, semantics: any[] | null }} */ (entries.get(main.folder)) + if (!mainLibrary.semantics) report('type', `${source.type} has no semantics: it is not a content type an author fills in`) + else if (!mainLibrary.library.runnable) report('type', `${mainLibrary.library.machineName} is a part other content holds, not a content type on its own: put it in one, such as H5P.Column`) + + // Layouts that adjust what is written, before it is generated. + const context = (/** @type {string} */ machineName) => { + const entry = [...entries.values()].find((each) => each.library.machineName === machineName && each.semantics) + return { + report, note, + semantics: entry?.semantics ?? [], + library: entry?.library, + title: (/** @type {string} */ library) => { + const name = String(library).split(' ')[0] + const found = [...entries.values()].find((each) => each.library.machineName === name && each.semantics) + return String(found?.library.title ?? name) + } + } + } + const written = structuredClone(source.params) + for (const place of placesOf(main.machineName, written)) LAYOUTS[place.library]?.before?.(place.params, place.path, context(place.library)) + + const content = generator.generate(mainLibrary, written) + problems.push(...generator.problems) + notes.push(...generator.notes) + + // Media: copied, described, put in place of what was written. + temporary = await mkdtemp(join(dirname(output), '.h5p-author-')) + const media = new MediaStore({ base: options.base ?? process.cwd(), temporary, fetch: options.fetch }) + for (const { field, value, path } of generator.media) { + const resolved = await media.resolve(field, value, path, report) + for (const key of Object.keys(value)) delete value[key] + Object.assign(value, resolved) + } + + // Layouts that set what the editor's widgets set, and check what they would not let pass. + for (const place of placesOf(main.machineName, content)) LAYOUTS[place.library]?.after?.(place.params, place.path, context(place.library)) + for (const place of placesOf(main.machineName, content)) { + if (place.holder && isObject(place.holder.metadata) && place.holder.metadata.title === undefined) place.holder.metadata.title = place.holder.metadata.contentType + } + for (const { field, path } of generator.unset) { + if (valueAt(content, path) === undefined && !problems.some((problem) => problem.path === path)) report(path, `is required (${field.type}), and nothing set it`) + } + if (problems.length) throw new SourceError(problems) + + const named = [libraryString(mainLibrary.library), ...[...librariesIn(content)].filter((name) => name !== libraryString(mainLibrary.library))] + const { selected, addons } = await closure(pack, named, content) + if (addons.length) notes.push({ path: 'params', message: `its text matches ${addons.join(', ')}, which the package carries, as a site with it installed would add it` }) + + // As H5P's exports write it, unescaped: the platforms check it when they load content. + const manifest = { + title: source.title, + language: source.language, + defaultLanguage: source.language, + license: source.license, + ...(source.licenseVersion ? { licenseVersion: source.licenseVersion } : {}), + authors: source.authors.map(({ name, role }) => ({ name, role })), + ...(source.source ? { source: source.source } : {}), + ...(source.yearFrom !== undefined ? { yearFrom: source.yearFrom } : {}), + ...(source.yearTo !== undefined ? { yearTo: source.yearTo } : {}), + mainLibrary: mainLibrary.library.machineName, + embedTypes: ['div'], + preloadedDependencies: [...selected.values()].map((part) => ({ + machineName: part.library.machineName, majorVersion: String(part.library.majorVersion), minorVersion: String(part.library.minorVersion) + })) + } + + const staged = join(temporary, 'package.h5p') + const file = await open(staged, 'wx') + try { + const writer = new ZipWriter(Writable.toWeb(file.createWriteStream()), { useWebWorkers: false }) + await writer.add('h5p.json', new TextReader(JSON.stringify(manifest))) + for (const part of selected.values()) { + await writer.appendZip(part.file, { filter: (entry) => !entry.directory && entry.filename.startsWith(`${part.folder}/`) }) + } + await writer.add('content/content.json', new TextReader(JSON.stringify(content))) + for (const item of media.files) { + await writer.add(`content/${item.name}`, /** @type {ReadableStream} */ (Readable.toWeb(createReadStream(item.from))), { level: 0 }) + } + await writer.close() + } finally { + await file.close() + } + let final = staged + if (media.files.length) { + // Media stored and last, an mp4's index first: the layout a package streams in. + final = join(temporary, 'normalized.h5p') + await normalizeArchive({ input: staged, output: final }) + } + // Unlike a hard link, an exclusive copy also works on FAT/exFAT destinations. + await copyFile(final, output, constants.COPYFILE_EXCL) + return { + output, + manifest, + content, + libraries: [...selected.keys()], + translations, + notes, + media: media.files.map((item) => item.name) + } + } finally { + await pack.close() + if (temporary) await rm(temporary, { recursive: true, force: true }) + } +} + +/** + * The value at a generator path, `params.a[0].b`, in the content. + * @param {any} content + * @param {string} path + */ +function valueAt(content, path) { + let value = { params: content } + for (const part of path.replace(/\[(\d+)\]/g, '.$1').split('.')) { + if (value === undefined || value === null) return undefined + value = /** @type {any} */ (value)[part] + } + return value +} diff --git a/packages/author/lib/generator.mjs b/packages/author/lib/generator.mjs new file mode 100644 index 00000000..4746d1ba --- /dev/null +++ b/packages/author/lib/generator.mjs @@ -0,0 +1,609 @@ +import { randomUUID } from 'node:crypto' +import { markupText, plainText } from './text.mjs' + +/** + * Content for a content type, made from its own `semantics.json` as H5P's editor saves a form: + * what is written where it says something, and everywhere else every default the semantics give. + * A field that holds another content type takes the library the written value names, filled in + * the same way. So a source states only what the defaults cannot know, and a list in it is as long + * as it is written. + * + * Two ways to run it. Authoring, the default, checks every written value against its field, as the + * editor's form would: a field the semantics do not name, a select's unknown option, a list too + * short or too long, a tag a markup field does not take, a required field with neither a written + * value nor a default — each a problem named by its path. The content-type sweep runs it + * `lenient`, taking what is written as it is and filling a required field from `fill`, so that + * every content type in the pack plays (apps/demo/scripts/sweep-content-types.mjs). + * + * Pure but for the ids: the pack is handed in as a map of folders, each with its `library.json` + * and `semantics.json`. Media are not read here: a written file is checked for its shape and + * collected in `media`, for the caller to copy and describe. + */ + +/** The nesting depth at which a library field stops choosing content of its own. */ +const MAX_DEPTH = 4 + +/** + * @typedef {{ folder: string, library: any, semantics: any[] | null }} PackEntry + * @typedef {{ path: string, message: string }} Problem + * @typedef {{ field: any, value: any, path: string }} MediaReference + * @typedef {{ + * lenient?: boolean, + * fill?: (field: any, context: { generator: ContentGenerator, stack: string[], path: string }) => unknown, + * id?: (count: number) => string, + * metadata?: (entry: PackEntry, written: any, path: string) => any + * }} GeneratorOptions + */ + +export class ContentGenerator { + /** + * @param {Map} pack every library folder the content may use, by folder name + * @param {GeneratorOptions} [options] + */ + constructor(pack, options = {}) { + this.pack = pack + this.options = options + this.lenient = options.lenient ?? false + this.ids = 0 + /** @type {Problem[]} what is wrong with the written content, authoring only */ + this.problems = [] + /** @type {Problem[]} what is worth saying about it and does not stop a build */ + this.notes = [] + /** @type {MediaReference[]} the files the written content names, for the caller to copy */ + this.media = [] + /** @type {WeakSet} fields a content type's own widget draws: numbers take decimals, lists start empty */ + this.drawn = new WeakSet() + /** @type {{ field: any, path: string }[]} hidden fields left for a content type's layout to set */ + this.unset = [] + /** @type {{ fields: any[], given: Record }[]} the groups being filled, innermost last, for `showWhen` */ + this.scopes = [] + } + + /** + * Whether the editor shows a field, by its `showWhen` rules: one of them holds, on the value + * written for the field they name, or its default. A field the editor hides is not asked for. + * @param {any} field + */ + #visible(field) { + const rules = field.showWhen?.rules + if (!Array.isArray(rules) || !rules.length) return true + const results = rules.map((/** @type {any} */ rule) => { + let name = String(rule.field ?? '') + let level = this.scopes.length - 1 + while (name.startsWith('../')) { + name = name.slice(3) + level -= 1 + } + const scope = this.scopes[level] + if (!scope) return true + const sibling = scope.fields.find((each) => each.name === name) + let value = scope.given[name] ?? sibling?.default + if (value === undefined && sibling?.type === 'boolean') value = false + if (isObject(value) && typeof value.library === 'string') value = value.library.trim().split(/\s+/)[0] + return [rule.equals].flat().some((/** @type {unknown} */ wanted) => { + if (typeof value === 'string' && typeof wanted === 'string' && /^\S+ \d+\.\d+$/.test(wanted)) return wanted.split(' ')[0] === value + return String(wanted) === String(value) + }) + }) + return field.showWhen.type === 'and' ? results.every(Boolean) : results.some(Boolean) + } + + /** A field as optional, for one the editor hides. @param {any} field */ + #optional(field) { + const optional = { ...field, optional: true } + if (this.drawn.has(field)) this.drawn.add(optional) + return optional + } + + /** + * The newest library in the pack with this machine name and, when given, this major — the + * one the player serves for any minor of it. + * @param {string} machineName + * @param {number} [major] + */ + find(machineName, major) { + /** @type {PackEntry | null} */ + let best = null + for (const entry of this.pack.values()) { + const { library } = entry + if (library.machineName !== machineName) continue + if (major !== undefined && library.majorVersion !== major) continue + if (!best || library.majorVersion > best.library.majorVersion || (library.majorVersion === best.library.majorVersion && library.minorVersion > best.library.minorVersion)) best = entry + } + return best + } + + /** + * The parameters for a library's content, with `written` — the authored parameters, if any — + * laid in. + * @param {PackEntry} entry + * @param {any} [written] + * @param {string} [path] where the parameters sit, for problems: `params` by default + */ + generate(entry, written, path = 'params') { + return this.fields(entry.semantics ?? [], written, [entry.library.machineName], false, path) + } + + /** A sub-content id: random, or the caller's, numbered, where it wants the same bytes every run. */ + nextId() { + this.ids += 1 + return this.options.id ? this.options.id(this.ids) : randomUUID() + } + + /** @param {string} path @param {string} message */ + report(path, message) { + this.problems.push({ path, message }) + } + + /** @param {string} path @param {string} message */ + note(path, message) { + this.notes.push({ path, message }) + } + + /** + * @param {any[]} fields + * @param {any} written the written object for these fields, or undefined + * @param {string[]} stack the content types this value sits inside, to keep a type out of itself + * @param {boolean} lenient inside an optional group nobody wrote: defaults only, as the editor + * saves a group left closed — and so nothing asked for a field required only within it + * @param {string} path + */ + fields(fields, written, stack, lenient, path) { + /** @type {Record} */ + const out = {} + if (!this.lenient && written !== undefined && !isObject(written)) { + this.report(path, `expected an object with the fields ${names(fields)}`) + return out + } + const given = isObject(written) ? written : {} + this.scopes.push({ fields, given }) + try { + for (const field of fields) { + const value = this.field(this.lenient || this.#visible(field) ? field : this.#optional(field), given[field.name], stack, lenient, `${path}.${field.name}`) + if (value !== undefined) out[field.name] = value + } + } finally { + this.scopes.pop() + } + for (const [key, value] of Object.entries(given)) { + if (key in out || fields.some((field) => field.name === key)) continue + // Lenient, what the semantics do not name is kept as written: the content check removes + // it, which a sweep file may mean to test. + if (this.lenient) { + if (value !== null) out[key] = value + } else this.report(`${path}.${key}`, `is not a field here; the fields are ${names(fields)}`) + } + return out + } + + /** + * @param {any} field + * @param {any} written the written value, `null` to leave the field out, or undefined + * @param {string[]} stack + * @param {boolean} lenient + * @param {string} path + * @returns {any} + */ + field(field, written, stack, lenient, path) { + const optional = field.optional || lenient + if (written === null) { + if (!this.lenient && !field.optional && field.default === undefined && field.type !== 'boolean') this.report(path, 'is required and cannot be null') + return undefined + } + switch (field.type) { + case 'group': { + const children = field.fields ?? [] + // A group of one field is saved as that field's value, as the editor and the content + // check have it, unless the group is sub-content. + const flattened = children.length === 1 && !field.isSubContent + if (written === undefined && field.default !== undefined) return structuredClone(field.default) + if (flattened) return this.field(children[0], written, stack, optional && written === undefined, path) + if (written !== undefined && !isObject(written)) { + if (this.lenient) return written + this.report(path, `expected an object with the fields ${names(children)}`) + return undefined + } + // A group the editor draws with a widget of its own — a duration, a position on an image, + // a slide — takes numbers as that widget sets them, decimals included, and starts its + // lists empty. + if (field.widget && field.widget !== 'showWhen' && field.widget !== 'none') this.#draw(children) + // Saved even when optional and untouched, as the editor saves every group in its form: + // content types read into one without looking (Personality Quiz's `image.file`). + return this.fields(children, written, stack, optional && written === undefined, path) + } + case 'list': { + const item = { ...field.field, optional: false } + if (Array.isArray(written)) { + if (!this.lenient) { + if (field.min !== undefined && written.length < field.min) this.report(path, `has ${written.length} item${written.length === 1 ? '' : 's'}; it takes at least ${field.min}`) + if (field.max !== undefined && written.length > field.max) this.report(path, `has ${written.length} items; it takes at most ${field.max}`) + } + return written.map((each, index) => this.field(item, each, stack, false, `${path}[${index}]`)).filter((each) => each !== undefined) + } + if (written !== undefined && !this.lenient) { + this.report(path, 'expected a list') + return undefined + } + const count = !this.lenient && this.drawn.has(field) + ? optional ? 0 : field.min ?? 0 + : Math.min(Math.max(optional ? 0 : (field.min ?? 0), field.defaultNum ?? (optional ? 0 : 1)), field.max ?? Infinity) + if (!this.lenient && count > 0) { + // A required list nobody wrote is what the editor offers a new author: as many items as + // it asks for, each with its defaults. Where an item needs something only an author can + // give, the list itself is what is missing. + const before = this.problems.length + const items = this.#items(item, count, stack, lenient, path) + if (this.problems.length > before) { + this.problems.length = before + this.report(path, `is required: write at least ${Math.max(field.min ?? 1, 1)} item${Math.max(field.min ?? 1, 1) === 1 ? '' : 's'}`) + return undefined + } + return items + } + const items = this.#items(item, count, stack, lenient, path) + return items.length > 0 || !optional ? items : undefined + } + case 'library': + if (isObject(written) && typeof written.library === 'string') return this.writtenLibrary(written, stack, path, field) + if (written !== undefined && !this.lenient) { + this.report(path, `expected sub-content, { "library": "${optionNames(field)[0] ?? 'H5P.…'}", "params": { … } }`) + return undefined + } + // A default names the content the editor starts with: Interactive Video's summary task. + // Left as the editor leaves it for an author who never opens it: what it would ask for + // is not asked here either, and stays unwritten. + if (written === undefined && isObject(field.default) && typeof field.default.library === 'string') { + if (this.lenient) return this.writtenLibrary(field.default, stack, path, null) + return this.#quietly(() => this.writtenLibrary(structuredClone(field.default), stack, path, null)) + } + return optional ? undefined : this.missing(field, stack, path) + case 'text': + if (written !== undefined) return this.lenient ? written : this.text(field, written, path) + // A default of null, which Course Presentation gives its colours, is stored as nothing: the + // platforms' check would turn it into an empty text. + if (field.default === null && !this.lenient) return undefined + if (field.default !== undefined) return field.default + return optional ? undefined : this.missing(field, stack, path) + case 'number': + if (written !== undefined) return this.lenient ? written : this.number(field, written, path) + if (field.default !== undefined) return field.default + return optional ? undefined : this.missing(field, stack, path) + case 'boolean': + if (written !== undefined && !this.lenient && typeof written !== 'boolean') this.report(path, 'expected true or false') + // The editor's checkbox saves false when nobody ticked it. + return written ?? field.default ?? false + case 'select': + if (written !== undefined) return this.lenient ? written : this.select(field, written, path) + if (field.default !== undefined) return structuredClone(field.default) + return optional ? undefined : this.missing(field, stack, path) + case 'image': + case 'file': + case 'video': + case 'audio': + if (written !== undefined) return this.lenient ? written : this.file(field, written, path) + return optional ? undefined : this.missing(field, stack, path) + default: + return written + } + } + + /** + * What a callback generates, with nothing it would report: problems, notes and hidden fields. + * @template T + * @param {() => T} generate + * @returns {T} + */ + #quietly(generate) { + const counts = [this.problems.length, this.notes.length, this.unset.length] + try { + return generate() + } finally { + this.problems.length = counts[0] + this.notes.length = counts[1] + this.unset.length = counts[2] + } + } + + /** Fields, and all within them, as a widget of the editor's draws them. @param {any[]} fields */ + #draw(fields) { + for (const field of fields) { + if (!field || this.drawn.has(field)) continue + this.drawn.add(field) + if (field.fields) this.#draw(field.fields) + if (field.field) this.#draw([field.field]) + } + } + + /** + * @param {any} item + * @param {number} count + * @param {string[]} stack + * @param {boolean} lenient + * @param {string} path + */ + #items(item, count, stack, lenient, path) { + const items = [] + for (let index = 0; index < count; index++) { + const value = this.field(item, undefined, stack, lenient, `${path}[${index}]`) + if (value !== undefined) items.push(value) + } + return items + } + + /** + * A required field with neither a written value nor a default: the caller's to fill, or a + * problem. + * @param {any} field + * @param {string[]} stack + * @param {string} path + */ + missing(field, stack, path) { + if (this.options.fill) return this.options.fill(field, { generator: this, stack, path }) + // A field the editor hides is its own bookkeeping, a position it sets when an author drags + // something: a content type's layout sets it here, and what none sets is a problem then. + if (field.widget === 'none') { + this.unset.push({ field, path }) + return undefined + } + this.report(path, `is required (${describe(field)})`) + return undefined + } + + /** + * Sub-content: the first option the pack has and that is not already a parent here, made from + * its own semantics. None at all is left out, which the content check would do anyway. The + * sweep's fill, for a field it has to fill to make a content type play. + * @param {any} field + * @param {string[]} stack + * @param {string} [path] + */ + library(field, stack, path = '') { + if (stack.length > MAX_DEPTH) return undefined + for (const option of field.options ?? []) { + const name = typeof option === 'string' ? option : option?.name + const match = /^(\S+) (\d+)\.(\d+)$/.exec(name ?? '') + if (!match || stack.includes(match[1])) continue + const entry = this.find(match[1], Number(match[2])) + if (!entry) continue + return this.subContent(entry, undefined, undefined, stack, path) + } + return undefined + } + + /** + * Written sub-content: the library it names, its parameters filled in from that library's + * semantics. Lenient, a library the pack does not have is kept as written, for the check to + * remove or the runtime to refuse; authoring, it is a problem, as is one the field does not take. + * @param {any} written + * @param {string[]} stack + * @param {string} path + * @param {any} field the library field, or null for a default the semantics give + */ + writtenLibrary(written, stack, path, field) { + if (this.lenient) { + const match = /^(\S+) (\d+)\.(\d+)$/.exec(written.library) + const entry = match ? this.find(match[1], Number(match[2])) : null + if (!entry) return written + return { ...this.subContent(entry, written.params, undefined, stack, path), ...omit(written, ['params']), library: libraryString(entry.library) } + } + const match = /^\s*(\S+?)(?:\s+(\d+)\.(\d+))?\s*$/.exec(written.library) + if (!match) { + this.report(`${path}.library`, `expected a library's machine name, such as ${optionNames(field)[0] ?? '"H5P.MultiChoice"'}`) + return undefined + } + const machineName = match[1] + let major = match[2] === undefined ? undefined : Number(match[2]) + if (field) { + const options = libraryOptions(field) + const option = options.find((each) => each.machineName === machineName && (major === undefined || each.major === major)) + if (!option) { + this.report(`${path}.library`, `${machineName} is not one this field takes; it takes ${[...new Set(options.map((each) => each.machineName))].join(', ')}`) + return undefined + } + major = option.major + } + const entry = this.find(machineName, major) + if (!entry) { + this.report(`${path}.library`, `the library pack has no ${machineName}${major === undefined ? '' : ` ${major}.x`}`) + return undefined + } + for (const key of Object.keys(written)) { + if (!['library', 'params', 'metadata'].includes(key)) this.report(`${path}.${key}`, 'is not part of sub-content; write library, params and metadata') + } + if (written.metadata !== undefined && !isObject(written.metadata)) this.report(`${path}.metadata`, 'expected an object, such as { "title": "…" }') + else if (isObject(written.metadata)) { + for (const key of Object.keys(written.metadata)) if (key !== 'title') this.report(`${path}.metadata.${key}`, 'is not set per sub-content here; write only its title, the rest comes from the source') + } + return this.subContent(entry, written.params, written.metadata, stack, path) + } + + /** + * @param {PackEntry} entry + * @param {any} written its parameters + * @param {any} metadata its written metadata + * @param {string[]} stack + * @param {string} path + */ + subContent(entry, written, metadata, stack, path) { + return { + library: libraryString(entry.library), + params: this.fields(entry.semantics ?? [], written, [...stack, entry.library.machineName], false, `${path}.params`), + subContentId: this.nextId(), + metadata: this.options.metadata + ? this.options.metadata(entry, metadata, path) + : { contentType: entry.library.title, license: 'U', title: entry.library.title } + } + } + + /** + * @param {any} field + * @param {unknown} written + * @param {string} path + */ + text(field, written, path) { + if (typeof written !== 'string') { + this.report(path, `expected text${typeof written === 'number' ? `, such as "${written}"` : ''}`) + return undefined + } + const stored = field.tags === undefined || field.tags === null + ? plainText(written, (message) => this.note(path, message)) + : markupText(written, field, (message) => this.report(path, message)) + const length = Array.from(stored).length + if (typeof field.maxLength === 'number' && length > field.maxLength) this.report(path, `is ${length} characters as stored; it takes at most ${field.maxLength}`) + if (field.regexp?.pattern && !(stored === '' && field.optional)) { + let pattern = null + try { + pattern = new RegExp(field.regexp.pattern, (field.regexp.modifiers ?? '').replace(/[^gimsuy]/g, '')) + } catch { + // A pattern JavaScript cannot read is the content check's to apply, not ours. + } + if (pattern && !pattern.test(stored)) this.report(path, `"${written.slice(0, 40)}" is not in the form this field takes${field.description ? ` (${stripTags(field.description).slice(0, 120)})` : ''}`) + } + if (!field.optional && stored.replace(/<[^>]*>/g, '').trim() === '' && !/<(img|iframe|video|audio|hr|table)/i.test(stored)) this.report(path, 'is required and is empty') + return stored + } + + /** + * @param {any} field + * @param {unknown} written + * @param {string} path + */ + number(field, written, path) { + if (typeof written !== 'number' || !Number.isFinite(written)) { + this.report(path, 'expected a number') + return undefined + } + if (field.min !== undefined && written < Number(field.min)) this.report(path, `is ${written}; the least it takes is ${field.min}`) + if (field.max !== undefined && written > Number(field.max)) this.report(path, `is ${written}; the most it takes is ${field.max}`) + if (field.decimals !== undefined && Number(field.decimals) >= 0) { + const factor = 10 ** Number(field.decimals) + if (Math.round(written * factor) / factor !== written) this.report(path, `takes ${field.decimals ? `at most ${field.decimals} decimals` : 'a whole number'}`) + } else if (!field.widget && !this.drawn.has(field) && !Number.isInteger(written)) this.report(path, 'takes a whole number') + if (field.step !== undefined && Number(field.step) > 0 && (written - Number(field.min ?? 0)) % Number(field.step) !== 0) this.report(path, `takes steps of ${field.step}`) + return written + } + + /** + * @param {any} field + * @param {unknown} written + * @param {string} path + */ + select(field, written, path) { + const values = selectValues(field) + const check = (/** @type {unknown} */ value) => { + if (values.length && !values.includes(String(value))) this.report(path, `${JSON.stringify(value)} is not an option; the options are ${values.map((each) => JSON.stringify(each)).join(', ')}`) + } + if (field.multiple) { + if (!Array.isArray(written)) this.report(path, 'expected a list of options') + else written.forEach(check) + } else if (typeof written !== 'string' && typeof written !== 'number' && typeof written !== 'boolean') this.report(path, 'expected one of its options') + else check(written) + return written + } + + /** + * A written file: `{ "path": … }`, a file beside the source or a web address, with `copyright` + * when known. A video or an audio field takes a list of them, one per format, or one alone. + * Checked for its shape here; the caller copies it and fills in its type and size. + * @param {any} field + * @param {unknown} written + * @param {string} path + */ + file(field, written, path) { + const many = field.type === 'video' || field.type === 'audio' + const list = many && Array.isArray(written) ? written : [written] + if (many && Array.isArray(written) && !written.length) { + this.report(path, 'expected at least one file') + return undefined + } + if (!many && Array.isArray(written)) { + this.report(path, 'expected one file, { "path": "…" }') + return undefined + } + const files = [] + for (const [index, each] of list.entries()) { + const at = many && Array.isArray(written) ? `${path}[${index}]` : path + if (!isObject(each) || typeof each.path !== 'string' || !each.path.trim()) { + this.report(at, `expected a file, { "path": "picture.jpg" }, a file beside the source or a web address`) + continue + } + for (const key of Object.keys(each)) if (!['path', 'copyright'].includes(key)) this.report(`${at}.${key}`, 'is not part of a file here; write path and, when known, copyright') + if (each.copyright !== undefined && !isObject(each.copyright)) this.report(`${at}.copyright`, 'expected an object, such as { "license": "CC BY", "author": "…" }') + const value = { ...each } + this.media.push({ field, value, path: at }) + files.push(value) + } + return many ? files : files[0] + } +} + +/** Every `library` string in the parameters: the sub-content a package plays. */ +export function librariesIn(/** @type {any} */ params, found = new Set()) { + if (Array.isArray(params)) for (const item of params) librariesIn(item, found) + else if (isObject(params)) { + if (typeof params.library === 'string' && /^\S+ \d+\.\d+$/.test(params.library)) found.add(params.library) + for (const value of Object.values(params)) librariesIn(value, found) + } + return found +} + +/** `H5P.MultiChoice 1.16`, the form `library` fields and `h5p.json` use. */ +export function libraryString(/** @type {any} */ library) { + return `${library.machineName} ${library.majorVersion}.${library.minorVersion}` +} + +/** @param {any} field @returns {{ machineName: string, major: number, minor: number }[]} */ +function libraryOptions(field) { + return (field?.options ?? []) + .map((/** @type {any} */ option) => /^(\S+) (\d+)\.(\d+)$/.exec(typeof option === 'string' ? option : option?.name ?? '')) + .filter(Boolean) + .map((/** @type {RegExpExecArray} */ match) => ({ machineName: match[1], major: Number(match[2]), minor: Number(match[3]) })) +} + +/** @param {any} field */ +function optionNames(field) { + return libraryOptions(field).map((option) => option.machineName) +} + +/** @param {any} field @returns {string[]} */ +function selectValues(field) { + /** @type {string[]} */ + const values = [] + for (const option of field.options ?? []) { + if (option?.type === 'optgroup') for (const sub of option.options ?? []) values.push(String(sub.value)) + else if (option?.value !== undefined && option?.value !== null) values.push(String(option.value)) + } + return values +} + +/** A field's kind, as a problem names it. @param {any} field */ +function describe(field) { + switch (field.type) { + case 'text': return field.tags ? 'markup text' : 'text' + case 'select': return `one of ${selectValues(field).map((value) => JSON.stringify(value)).join(', ')}` + case 'library': return `sub-content: ${optionNames(field).join(', ')}` + case 'image': case 'file': return `a file, { "path": "…" }` + case 'video': case 'audio': return `a list of files, [{ "path": "…" }]` + default: return field.type + } +} + +/** @param {any[]} fields */ +function names(fields) { + return fields.map((field) => field.name).join(', ') +} + +/** @param {string} html */ +function stripTags(html) { + return html.replace(/<[^>]*>/g, ' ').replace(/\s+/g, ' ').trim() +} + +/** @param {any} object @param {string[]} keys */ +function omit(object, keys) { + return Object.fromEntries(Object.entries(object).filter(([key]) => !keys.includes(key))) +} + +/** @param {unknown} value @returns {value is Record} */ +function isObject(value) { + return value !== null && typeof value === 'object' && !Array.isArray(value) +} diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs new file mode 100644 index 00000000..eb483cfc --- /dev/null +++ b/packages/author/lib/media.mjs @@ -0,0 +1,307 @@ +import { createWriteStream } from 'node:fs' +import { open, stat } from 'node:fs/promises' +import { basename, extname, join, resolve } from 'node:path' +import { Readable, Transform } from 'node:stream' +import { pipeline } from 'node:stream/promises' +import { escapeText } from './text.mjs' + +/** + * The files content shows: images, video, audio and other files, named in a source by a path + * beside it or a web address, copied into the package's `content/` folder with what H5P records + * of each — its type, an image's size, its copyright. A video on YouTube, Vimeo or Panopto stays + * there and is played from there, as H5P's editor embeds it. + */ + +/** The types H5P accepts for content (`H5PCore::$defaultContentWhitelist`), by what a field takes. */ +const TYPES = { + image: { png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', gif: 'image/gif', bmp: 'image/bmp' }, + video: { mp4: 'video/mp4', webm: 'video/webm', ogg: 'video/ogg' }, + audio: { mp3: 'audio/mpeg', m4a: 'audio/mp4', wav: 'audio/wav', ogg: 'audio/ogg', webm: 'audio/webm' }, + file: { + pdf: 'application/pdf', txt: 'text/plain', csv: 'text/csv', md: 'text/markdown', vtt: 'text/vtt', webvtt: 'text/vtt', rtf: 'application/rtf', + doc: 'application/msword', docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', + xls: 'application/vnd.ms-excel', xlsx: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', + ppt: 'application/vnd.ms-powerpoint', pptx: 'application/vnd.openxmlformats-officedocument.presentationml.presentation', + odt: 'application/vnd.oasis.opendocument.text', ods: 'application/vnd.oasis.opendocument.spreadsheet', odp: 'application/vnd.oasis.opendocument.presentation', + png: 'image/png', jpg: 'image/jpeg', jpeg: 'image/jpeg', gif: 'image/gif', mp3: 'audio/mpeg', mp4: 'video/mp4', json: 'application/json' + } +} + +/** The folder under `content/` H5P's editor keeps each kind in. */ +const FOLDERS = { image: 'images', video: 'videos', audio: 'audios', file: 'files' } + +/** Video a host plays itself, which H5P.Video embeds by address. */ +const HOSTED = [ + { pattern: /^https?:\/\/(?:www\.|m\.)?(?:youtube\.com\/(?:watch\?|embed\/|shorts\/)|youtu\.be\/)/i, mime: 'video/YouTube' }, + { pattern: /^https?:\/\/(?:www\.|player\.)?vimeo\.com\//i, mime: 'video/Vimeo' }, + { pattern: /^https?:\/\/[^/]+\.panopto\.(?:com|eu)\//i, mime: 'video/Panopto' } +] + +/** The licences a file's copyright may name, as H5P lists them. */ +const COPYRIGHT_LICENSES = ['U', 'CC BY', 'CC BY-SA', 'CC BY-ND', 'CC BY-NC', 'CC BY-NC-SA', 'CC BY-NC-ND', 'GNU GPL', 'PD', 'C'] + +/** The most a download may bring: a package of this size is already past what a learner waits for. */ +const MAX_DOWNLOAD = 1024 ** 3 + +/** + * @typedef {{ name: string, from: string }} StagedFile a file to add to the package: its name under `content/`, and where it is on disk + * @typedef {{ path: string, message: string }} Problem + */ + +/** + * An image's width and height, read from its header: PNG, JPEG, GIF and BMP. + * @param {Uint8Array} bytes the first bytes of the file, 64 kB or the whole of a smaller one + * @returns {{ width: number, height: number } | null} + * @internal + */ +export function imageSize(bytes) { + const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength) + const ascii = (/** @type {number} */ offset, /** @type {number} */ length) => String.fromCharCode(...bytes.subarray(offset, offset + length)) + if (bytes.length >= 24 && ascii(1, 3) === 'PNG' && ascii(12, 4) === 'IHDR') return { width: view.getUint32(16), height: view.getUint32(20) } + if (bytes.length >= 10 && (ascii(0, 6) === 'GIF87a' || ascii(0, 6) === 'GIF89a')) return { width: view.getUint16(6, true), height: view.getUint16(8, true) } + if (bytes.length >= 26 && ascii(0, 2) === 'BM') return { width: Math.abs(view.getInt32(18, true)), height: Math.abs(view.getInt32(22, true)) } + if (bytes.length >= 4 && bytes[0] === 0xff && bytes[1] === 0xd8) { + let offset = 2 + while (offset + 9 < bytes.length) { + if (bytes[offset] !== 0xff) return null + const marker = bytes[offset + 1] + if (marker === 0xff) { + offset += 1 + continue + } + const length = view.getUint16(offset + 2) + // A start-of-frame marker, baseline to lossless, holds the size; DHT, JPG and DAC do not. + if (marker >= 0xc0 && marker <= 0xcf && marker !== 0xc4 && marker !== 0xc8 && marker !== 0xcc) { + return { width: view.getUint16(offset + 7), height: view.getUint16(offset + 5) } + } + offset += 2 + length + } + } + return null +} + +/** + * Whether an mp4 has its index (`moov`) before its media (`mdat`), so it plays before it has all + * arrived. Read from the top-level boxes; an unreadable file counts as fine, since the normalizer + * leaves what it cannot read alone. + * @param {string} file + * @internal + */ +export async function isFaststart(file) { + const handle = await open(file, 'r') + try { + const { size } = await handle.stat() + let offset = 0 + const header = new Uint8Array(16) + while (offset + 8 <= size) { + await handle.read(header, 0, 16, offset) + const view = new DataView(header.buffer) + let length = view.getUint32(0) + const type = String.fromCharCode(...header.subarray(4, 8)) + if (length === 1) length = Number(view.getBigUint64(8)) + else if (length === 0) length = size - offset + if (type === 'moov') return true + if (type === 'mdat') return false + if (length < 8) return true + offset += length + } + return true + } finally { + await handle.close() + } +} + +export class MediaStore { + /** + * @param {{ base: string, temporary: string, fetch?: typeof fetch }} options + * `base`, the folder a source's relative paths are read from; `temporary`, where downloads land + */ + constructor({ base, temporary, fetch: fetchImpl = globalThis.fetch }) { + this.base = base + this.temporary = temporary + this.fetch = fetchImpl + /** @type {StagedFile[]} */ + this.files = [] + /** @type {Map} by source, so one file named twice is copied once */ + this.known = new Map() + /** @type {Set} */ + this.names = new Set() + this.downloads = 0 + /** Whether a video has its index at the end, which the build then moves to the front. */ + this.slowStart = false + } + + /** + * A written file made what H5P stores: `{ path, mime, width?, height?, copyright }`, its bytes + * staged for the package. + * @param {any} field the image, video, audio or file field + * @param {{ path: string, copyright?: any }} value as written + * @param {string} at where it is written, for problems + * @param {(path: string, message: string) => void} report + * @returns {Promise>} + */ + async resolve(field, value, at, report) { + const kind = /** @type {'image' | 'video' | 'audio' | 'file'} */ (field.type) + const source = value.path.trim() + const copyright = this.copyright(value.copyright, `${at}.copyright`, report) + const web = /^https?:\/\//i.test(source) + if (web && kind === 'video') { + const hosted = HOSTED.find(({ pattern }) => pattern.test(source)) + if (hosted) return { path: source, mime: hosted.mime, copyright } + } + if (/^[a-z][a-z0-9+.-]*:/i.test(source) && !web) { + report(`${at}.path`, 'expected a file beside the source or an http(s) address') + return { path: source, copyright } + } + const types = /** @type {Record} */ (TYPES[kind]) + let known = this.known.get(source) + if (!known) { + const staged = web ? await this.#download(source, kind, types, at, report) : await this.#local(source, types, at, report) + if (!staged) return { path: source, copyright } + const extension = staged.extension + const mime = types[extension] + const name = this.#name(FOLDERS[kind], staged.stem, extension) + /** @type {{ name: string, mime: string, width?: number, height?: number }} */ + const described = { name, mime } + if (kind === 'image') { + const handle = await open(staged.from, 'r') + try { + const head = new Uint8Array(65536) + const { bytesRead } = await handle.read(head, 0, head.length, 0) + const size = imageSize(head.subarray(0, bytesRead)) + if (!size) report(`${at}.path`, `${source} is not a ${extension.toUpperCase()} image whose size can be read`) + else Object.assign(described, size) + } finally { + await handle.close() + } + } + if (kind === 'video' && mime === 'video/mp4' && !(await isFaststart(staged.from))) this.slowStart = true + this.files.push({ name, from: staged.from }) + this.known.set(source, described) + known = described + } + const { name, ...rest } = known + return { path: name, ...rest, copyright } + } + + /** + * A file's copyright as H5P records it: a licence, and its author, title, year, source and + * version when given, as plain text. + * @param {unknown} written + * @param {string} at + * @param {(path: string, message: string) => void} report + */ + copyright(written, at, report) { + if (written === undefined) return { license: 'U' } + if (!written || typeof written !== 'object' || Array.isArray(written)) return { license: 'U' } + const value = /** @type {Record} */ (written) + /** @type {Record} */ + const out = { license: 'U' } + for (const [key, entry] of Object.entries(value)) { + if (!['license', 'author', 'title', 'year', 'source', 'version'].includes(key)) { + report(`${at}.${key}`, 'is not part of a copyright; write license, author, title, year, source or version') + continue + } + if (typeof entry !== 'string' || !entry.trim()) { + report(`${at}.${key}`, 'expected text') + continue + } + if (key === 'license' && !COPYRIGHT_LICENSES.includes(entry)) report(`${at}.license`, `use one of ${COPYRIGHT_LICENSES.join(', ')}`) + if (key === 'source' && !/^https?:\/\/.+/i.test(entry)) report(`${at}.source`, 'expected an http(s) address') + out[key] = key === 'license' || key === 'version' ? entry : escapeText(entry) + } + return out + } + + /** + * @param {string} source + * @param {Record} types + * @param {string} at + * @param {(path: string, message: string) => void} report + */ + async #local(source, types, at, report) { + const from = resolve(this.base, source) + const extension = extname(from).slice(1).toLowerCase() + if (!(extension in types)) { + report(`${at}.path`, `${source}: this field takes ${Object.keys(types).map((each) => `.${each}`).join(', ')}`) + return null + } + try { + const info = await stat(from) + if (!info.isFile()) throw new Error('not a file') + } catch { + report(`${at}.path`, `${source}: no such file beside the source (${from})`) + return null + } + return { from, extension, stem: basename(from, extname(from)) } + } + + /** + * A web address downloaded to the temporary folder, its type taken from its name or, failing + * that, from what the server says it is. + * @param {string} source + * @param {string} kind + * @param {Record} types + * @param {string} at + * @param {(path: string, message: string) => void} report + */ + async #download(source, kind, types, at, report) { + let url + try { + url = new URL(source) + } catch { + report(`${at}.path`, `${source} is not a web address`) + return null + } + let response + try { + response = await this.fetch(url.href, { redirect: 'follow', signal: AbortSignal.timeout(120_000) }) + } catch (error) { + report(`${at}.path`, `${source} could not be downloaded: ${error instanceof Error ? error.message : error}`) + return null + } + if (!response.ok || !response.body) { + report(`${at}.path`, `${source} answered ${response.status}${kind === 'video' ? '; for YouTube, Vimeo or Panopto give the video page\'s address' : ''}`) + return null + } + const named = extname(new URL(response.url || url.href).pathname).slice(1).toLowerCase() + const served = (response.headers.get('content-type') ?? '').split(';')[0].trim().toLowerCase() + const extension = named in types ? named : Object.keys(types).find((each) => types[each] === served) + if (!extension) { + await response.body.cancel() + report(`${at}.path`, `${source} is ${served || 'of an unknown type'}: this field takes ${Object.keys(types).map((each) => `.${each}`).join(', ')}`) + return null + } + this.downloads += 1 + const from = join(this.temporary, `download-${this.downloads}.${extension}`) + let received = 0 + const limit = new Transform({ + transform(chunk, _encoding, callback) { + received += chunk.length + callback(received > MAX_DOWNLOAD ? new Error(`larger than ${MAX_DOWNLOAD / 1024 ** 3} GB`) : null, chunk) + } + }) + try { + await pipeline(Readable.fromWeb(/** @type {import('node:stream/web').ReadableStream} */ (response.body)), limit, createWriteStream(from)) + } catch (error) { + report(`${at}.path`, `${source} could not be downloaded: ${error instanceof Error ? error.message : error}`) + return null + } + const stem = basename(decodeURIComponent(new URL(response.url || url.href).pathname), `.${named}`) || kind + return { from, extension, stem } + } + + /** + * A name under `content/` for a file: its own, made safe, and numbered where another has it. + * @param {string} folder + * @param {string} stem + * @param {string} extension + */ + #name(folder, stem, extension) { + const safe = stem.normalize('NFKD').replace(/[^\w.-]+/g, '-').replace(/^[-.]+|[-.]+$/g, '').slice(0, 60) || folder.slice(0, -1) + let name = `${folder}/${safe}.${extension}` + for (let count = 2; this.names.has(name.toLowerCase()); count++) name = `${folder}/${safe}-${count}.${extension}` + this.names.add(name.toLowerCase()) + return name + } +} diff --git a/packages/author/lib/pack.mjs b/packages/author/lib/pack.mjs new file mode 100644 index 00000000..ef05a655 --- /dev/null +++ b/packages/author/lib/pack.mjs @@ -0,0 +1,214 @@ +import { createHash } from 'node:crypto' +import { createReadStream } from 'node:fs' +import { open, readFile } from 'node:fs/promises' +import { join, resolve } from 'node:path' +import { Readable } from 'node:stream' +import { fileURLToPath, pathToFileURL } from 'node:url' +import { Reader, TextWriter, ZipReader } from '@zip.js/zip.js' + +/** + * The library pack a build takes its libraries from: by default the one this package was released + * with, `@missing-elements/h5p-libraries` at the version it pins; or a copy of its `parts/` folder. + * Its index says each library's version, dependencies and part; a part is opened only when a + * build needs that library, and checked against the index's SHA-256 first, so a damaged or + * altered part is never read, let alone copied into a package. + */ + +/** + * @typedef {{ + * folder: string, machineName: string, majorVersion: number, minorVersion: number, patchVersion: number, + * preloadedDependencies?: { machineName: string, majorVersion: number, minorVersion: number }[], + * addTo?: any, size: number, sha256: string, url: string + * }} IndexEntry + */ + +/** + * A library whose part has been opened: what the generator reads (`folder`, `library`, + * `semantics`) and what a build copies. + * @typedef {{ + * folder: string, library: any, semantics: any[] | null, index: IndexEntry, + * translations: Map, file: PartFile, reader: ZipReader + * }} PartLibrary + */ + +/** + * A part read from disk by range: the largest, MathDisplay's, need not be held whole. + * @extends {Reader} + */ +class PartFile extends Reader { + /** @param {string} path */ + constructor(path) { + super(path) + this.path = path + /** @type {import('node:fs/promises').FileHandle | null} */ + this.handle = null + } + + async init() { + await super.init?.() + this.handle = await open(this.path, 'r') + this.size = (await this.handle.stat()).size + } + + /** + * A span streamed from disk, which zip.js copies an entry through. + * @param {{ offset?: number, size?: number }} [span] + * @returns {ReadableStream} + */ + createReadable({ offset = 0, size = this.size - offset } = {}) { + if (!size) return new ReadableStream({ start(controller) { controller.close() } }) + return /** @type {ReadableStream} */ (Readable.toWeb(createReadStream(this.path, { start: offset, end: offset + size - 1 }))) + } + + /** @param {number} offset @param {number} length */ + async readUint8Array(offset, length) { + if (!this.handle) throw new Error(`Library part not open: ${this.path}`) + const bytes = new Uint8Array(length) + let read = 0 + while (read < length) { + const { bytesRead } = await this.handle.read(bytes, read, length - read, offset + read) + if (!bytesRead) throw new Error(`Short read from library part: ${this.path}`) + read += bytesRead + } + return bytes + } + + async close() { + await this.handle?.close() + this.handle = null + } +} + +export class Pack { + /** + * @param {IndexEntry[]} libraries + * @param {string} origin where the pack came from, for messages + */ + constructor(libraries, origin) { + this.libraries = libraries + this.origin = origin + /** @type {Map>} */ + this.opened = new Map() + } + + /** + * The library the pack serves for a machine name, and a major when given: the newest, as the + * player serves it for any minor of that major. + * @param {string} machineName + * @param {number} [major] + */ + newest(machineName, major) { + let best = null + for (const entry of this.libraries) { + if (entry.machineName !== machineName || (major !== undefined && entry.majorVersion !== major)) continue + if (!best || entry.majorVersion > best.majorVersion || (entry.majorVersion === best.majorVersion && (entry.minorVersion > best.minorVersion || (entry.minorVersion === best.minorVersion && entry.patchVersion > best.patchVersion)))) best = entry + } + return best + } + + /** + * A library's part, opened once: checked against the index, then its `library.json`, + * `semantics.json` and the names of its translations read. + * @param {IndexEntry} entry + * @returns {Promise} + */ + open(entry) { + let opened = this.opened.get(entry.folder) + if (!opened) { + opened = this.#open(entry) + this.opened.set(entry.folder, opened) + } + return opened + } + + /** @param {IndexEntry} entry */ + async #open(entry) { + const path = fileURLToPath(entry.url) + const hash = createHash('sha256') + try { + for await (const chunk of createReadStream(path)) hash.update(chunk) + } catch (error) { + throw new Error(`Cannot read the library part ${entry.folder} from ${this.origin}`, { cause: error }) + } + if (hash.digest('hex') !== entry.sha256) throw new Error(`The library part ${entry.folder} does not match the pack's index (SHA-256): reinstall the pack.`) + const file = new PartFile(path) + const reader = new ZipReader(file, { checkSignature: true }) + const entries = (await reader.getEntries()).filter((item) => !item.directory && item.filename.startsWith(`${entry.folder}/`)) + /** @param {string} name */ + const json = async (name) => { + const found = entries.find((item) => item.filename === `${entry.folder}/${name}`) + return found ? JSON.parse(await /** @type {import('@zip.js/zip.js').FileEntry} */ (found).getData(new TextWriter())) : null + } + const library = await json('library.json') + if (!library) throw new Error(`The library part ${entry.folder} has no library.json`) + if (library.machineName !== entry.machineName || Number(library.majorVersion) !== entry.majorVersion || Number(library.minorVersion) !== entry.minorVersion || Number(library.patchVersion) !== entry.patchVersion) { + throw new Error(`The library part ${entry.folder} disagrees with the pack's index`) + } + const languages = `${entry.folder}/language/` + const translations = new Map( + entries + .filter((item) => item.filename.startsWith(languages) && item.filename.endsWith('.json')) + .map((item) => [item.filename.slice(languages.length, -'.json'.length), /** @type {import('@zip.js/zip.js').FileEntry} */ (item)]) + ) + return { folder: entry.folder, library, semantics: await json('semantics.json'), index: entry, translations, file, reader } + } + + /** + * A translation file of an opened library, parsed: its `semantics`. + * @param {PartLibrary} part + * @param {string} name + */ + async translation(part, name) { + const file = part.translations.get(name) + if (!file) return undefined + try { + return JSON.parse(await file.getData(new TextWriter())).semantics + } catch (error) { + throw new Error(`The translation ${file.filename} is not JSON`, { cause: error }) + } + } + + /** Every opened part closed. */ + async close() { + for (const opened of this.opened.values()) { + try { + const part = await opened + try { + await part.reader.close() + } finally { + await part.file.close() + } + } catch { + // A part that failed to open has nothing to close. + } + } + this.opened.clear() + } +} + +/** + * The pack a build reads: the one this package pins, or the `parts/` folder of another copy. + * @param {string} [folder] a folder holding `index.json` and the parts it names + */ +export async function openPack(folder) { + if (folder) { + const directory = resolve(folder) + let index + try { + index = JSON.parse(await readFile(join(directory, 'index.json'), 'utf8')) + } catch (error) { + throw new Error(`${directory} is not a library pack's parts folder: it has no readable index.json`, { cause: error }) + } + if (!Array.isArray(index?.libraries)) throw new Error(`${join(directory, 'index.json')} lists no libraries`) + const libraries = index.libraries.map((/** @type {IndexEntry} */ entry) => ({ ...entry, url: pathToFileURL(join(directory, `${entry.folder}.h5p`)).href })) + return new Pack(libraries, directory) + } + const { pack } = await import('@missing-elements/h5p-libraries') + return new Pack(/** @type {IndexEntry[]} */ (pack.libraries), '@missing-elements/h5p-libraries') +} + +/** The pinned pack's version, for messages and reports. */ +export async function packVersion() { + const url = import.meta.resolve('@missing-elements/h5p-libraries/package.json') + return JSON.parse(await readFile(fileURLToPath(url), 'utf8')).version +} diff --git a/packages/author/lib/quiz.mjs b/packages/author/lib/quiz.mjs new file mode 100644 index 00000000..44eac8e1 --- /dev/null +++ b/packages/author/lib/quiz.mjs @@ -0,0 +1,149 @@ +import { escapeText } from './text.mjs' + +/** + * The quiz format the h5p-author skill published on 2026-10-23: a Multiple Choice question or a + * Question Set of them, in plain text. It stays a source the build takes, mapped here onto the + * general one, so nothing written in it breaks. + */ + +/** + * @typedef {{ text: string, correct: boolean, feedback?: string }} Answer + * @typedef {{ title?: string, prompt: string, multipleAnswers?: boolean, answers: Answer[] }} Question + * @typedef {{ + * kind: 'multiple-choice' | 'question-set', title: string, language: string, license: string, + * authors: { name: string, role: 'Author' }[], passPercentage?: number, + * passedFeedback?: string, failedFeedback?: string, questions: Question[] + * }} Quiz + */ + +/** @param {unknown} value @param {string} path @param {string[]} keys */ +function record(value, path, keys) { + if (!value || typeof value !== 'object' || Array.isArray(value)) throw new Error(`${path}: expected an object`) + for (const key of Object.keys(value)) { + if (!keys.includes(key)) throw new Error(`${path}.${key}: unsupported field`) + } +} + +/** @param {unknown} value @param {string} path */ +function text(value, path) { + if (typeof value !== 'string' || !value.trim()) throw new Error(`${path}: expected non-empty text`) +} + +/** @param {unknown} source @returns {asserts source is Quiz} @internal */ +export function validateQuiz(source) { + record(source, 'quiz', ['kind', 'title', 'language', 'license', 'authors', 'passPercentage', 'passedFeedback', 'failedFeedback', 'questions']) + const quiz = /** @type {Quiz} */ (source) + if (!['multiple-choice', 'question-set'].includes(quiz.kind)) throw new Error('quiz.kind: use "multiple-choice" or "question-set"') + for (const key of /** @type {const} */ (['title', 'language', 'license'])) text(quiz[key], `quiz.${key}`) + if (quiz.title.length > 255) throw new Error('quiz.title: at most 255 characters') + try { + Intl.getCanonicalLocales(quiz.language) + } catch { + throw new Error('quiz.language: expected a BCP 47 language tag, such as "en"') + } + if (!['U', 'CC BY', 'CC BY-SA', 'CC BY-ND', 'CC BY-NC', 'CC BY-NC-SA', 'CC BY-NC-ND', 'CC0 1.0', 'PD', 'C'].includes(quiz.license)) { + throw new Error('quiz.license: use an H5P license code (for example "U", "CC BY", or "CC0 1.0")') + } + if (!Array.isArray(quiz.authors) || !quiz.authors.length) throw new Error('quiz.authors: name at least one author') + quiz.authors.forEach((author, i) => { + record(author, `quiz.authors[${i}]`, ['name', 'role']) + text(author.name, `quiz.authors[${i}].name`) + if (author.role !== 'Author') throw new Error(`quiz.authors[${i}].role: use "Author"`) + }) + if (quiz.passPercentage !== undefined && (!Number.isInteger(quiz.passPercentage) || quiz.passPercentage < 0 || quiz.passPercentage > 100)) { + throw new Error('quiz.passPercentage: expected an integer from 0 to 100') + } + if (quiz.passedFeedback !== undefined || quiz.failedFeedback !== undefined) { + // One without the other leaves some learners with no word on the result. + for (const key of /** @type {const} */ (['passedFeedback', 'failedFeedback'])) { + if (quiz[key] === undefined) throw new Error(`quiz.${key}: set passedFeedback and failedFeedback together`) + text(quiz[key], `quiz.${key}`) + } + if (quiz.passPercentage === undefined) throw new Error('quiz.passPercentage: set it with passedFeedback and failedFeedback; the library defaults differ by kind') + } + if (!Array.isArray(quiz.questions) || !quiz.questions.length) throw new Error('quiz.questions: at least one question is required') + if (quiz.kind === 'multiple-choice' && quiz.questions.length !== 1) throw new Error('quiz.questions: multiple-choice requires exactly one question') + quiz.questions.forEach((question, i) => { + const path = `quiz.questions[${i}]` + record(question, path, ['title', 'prompt', 'multipleAnswers', 'answers']) + text(question.prompt, `${path}.prompt`) + if (question.title !== undefined) text(question.title, `${path}.title`) + if (question.multipleAnswers !== undefined && typeof question.multipleAnswers !== 'boolean') throw new Error(`${path}.multipleAnswers: expected a boolean`) + if (!Array.isArray(question.answers) || question.answers.length < 2) throw new Error(`${path}.answers: at least two answers are required`) + question.answers.forEach((answer, j) => { + const answerPath = `${path}.answers[${j}]` + record(answer, answerPath, ['text', 'correct', 'feedback']) + text(answer.text, `${answerPath}.text`) + if (typeof answer.correct !== 'boolean') throw new Error(`${answerPath}.correct: expected a boolean`) + if (answer.feedback !== undefined) text(answer.feedback, `${answerPath}.feedback`) + }) + const correct = question.answers.filter((answer) => answer.correct).length + if (question.multipleAnswers ? correct === 0 : correct !== 1) { + throw new Error(`${path}.answers: ${question.multipleAnswers ? 'at least one' : 'exactly one'} answer must be correct`) + } + }) +} + +/** A plain text written as the paragraph the quiz format has always made of it. @param {string} value */ +function paragraph(value) { + return `

${escapeText(value).replaceAll('\n', '
')}

` +} + +/** + * The pass mark as two score ranges for the content's overall feedback. The libraries pick a + * range by the floored percentage and judge `success` by the exact one, which agree for a whole + * number, so the message a learner gets matches the result an LMS records. + * @param {Quiz} quiz + */ +function passFeedback(quiz) { + if (quiz.passedFeedback === undefined || quiz.failedFeedback === undefined || quiz.passPercentage === undefined) return undefined + const pass = quiz.passPercentage + return [ + ...(pass > 0 ? [{ from: 0, to: pass - 1, feedback: quiz.failedFeedback }] : []), + { from: pass, to: 100, feedback: quiz.passedFeedback } + ] +} + +/** + * A quiz as a general source: Multiple Choice's parameters for each question, inside a Question + * Set's for a set. A question with no title takes the set's own label and number, as the set's + * layout does (`H5P.QuestionSet` in types.mjs). + * @param {Quiz} quiz + * @internal + */ +export function quizSource(quiz) { + const overallFeedback = passFeedback(quiz) + const set = quiz.kind === 'question-set' + const questions = quiz.questions.map((question) => ({ + question: paragraph(question.prompt), + answers: question.answers.map((answer) => ({ + text: paragraph(answer.text), + correct: answer.correct, + tipsAndFeedback: answer.feedback ? { chosenFeedback: paragraph(answer.feedback) } : {} + })), + behaviour: { + type: question.multipleAnswers ? 'multi' : 'single', + ...(!set && quiz.passPercentage !== undefined ? { passPercentage: quiz.passPercentage } : {}) + }, + ...(!set && overallFeedback ? { overallFeedback } : {}) + })) + const params = set + ? { + questions: questions.map((params, i) => ({ + library: 'H5P.MultiChoice', + params, + ...(quiz.questions[i].title === undefined ? {} : { metadata: { title: quiz.questions[i].title } }) + })), + ...(quiz.passPercentage === undefined ? {} : { passPercentage: quiz.passPercentage }), + ...(overallFeedback ? { endGame: { overallFeedback } } : {}) + } + : questions[0] + return { + type: set ? 'H5P.QuestionSet' : 'H5P.MultiChoice', + title: quiz.title, + language: quiz.language, + license: quiz.license, + authors: quiz.authors, + params + } +} diff --git a/packages/author/lib/schema.mjs b/packages/author/lib/schema.mjs new file mode 100644 index 00000000..711ccd25 --- /dev/null +++ b/packages/author/lib/schema.mjs @@ -0,0 +1,224 @@ +import { allowedTags } from './text.mjs' + +/** + * A content type's fields as an author writes them, from its `semantics.json`: names, kinds, + * which are required, defaults, options, list sizes, the tags a markup field takes, the sub-content + * a library field takes, and the syntax the editor explains for a field. What an author never + * writes is left out: the interface texts, which come from the libraries' translations, and the + * hidden fields a content type's layout sets. A group of one field is shown as that field, since + * it is written as that field's value. + */ + +/** Groups that hold interface texts, under the names the pack's libraries give them. */ +const INTERFACE_GROUPS = new Set(['l10n', 'i10n', 'a11y', 'texts', 'UI', 'localize', 'labels', 'translations']) + +/** + * Whether a field is an interface text, or a group of nothing else: what a translation supplies. + * @param {any} field + * @returns {boolean} + */ +export function isInterfaceText(field) { + if (field.type === 'text') return Boolean(field.common) + if (field.type === 'group') { + const children = field.fields ?? [] + const texts = children.length > 0 && children.every((/** @type {any} */ child) => child.type === 'text' || (child.type === 'group' && isInterfaceText(child))) + return texts && (Boolean(field.common) || INTERFACE_GROUPS.has(field.name)) + } + return false +} + +/** @param {string} html */ +function plain(html) { + return html.replace(/<\/li>/g, '. ').replace(/<[^>]*>/g, ' ').replace(/ /g, ' ').replace(/"/g, '"').replace(/'|'/g, "'").replace(/>/g, '>').replace(/</g, '<').replace(/&/g, '&').replace(/\s+/g, ' ').replace(/\.\s*\./g, '.').trim() +} + +/** @param {unknown} value */ +function short(value) { + const text = JSON.stringify(value) + return text.length > 48 ? `${text.slice(0, 45)}…` : text +} + +/** @param {any} field @returns {string[]} */ +function selectValues(field) { + /** @type {string[]} */ + const values = [] + for (const option of field.options ?? []) { + if (option?.type === 'optgroup') for (const sub of option.options ?? []) values.push(String(sub.value)) + else if (option?.value !== undefined && option?.value !== null) values.push(String(option.value)) + } + return values +} + +/** + * @typedef {{ has?: (machineName: string) => boolean, sets?: string[], descriptions?: boolean, all?: boolean }} SchemaOptions + * `has`, whether the pack has a library, to list only the sub-content it can build; `sets`, the + * hidden fields a layout sets, as `a.b[].c`; `descriptions`, each field's description; `all`, + * the interface texts too + */ + +/** + * What a field takes, in a few words. + * @param {any} field + * @param {SchemaOptions} options + * @param {boolean} [drawn] drawn by a widget, whose numbers take decimals + */ +function kind(field, options, drawn = false) { + switch (field.type) { + case 'text': { + const tags = allowedTags(field) + if (!tags) return field.widget === 'textarea' ? 'text, multi-line' : 'text' + const own = (Array.isArray(field.tags) ? field.tags : []).filter((/** @type {string} */ tag) => !['div', 'span', 'p', 'br'].includes(tag)) + return own.length ? `html: ${own.join(' ')}` : 'html' + } + case 'number': { + const range = field.min !== undefined || field.max !== undefined ? ` ${field.min ?? ''}–${field.max ?? ''}` : '' + return `${field.decimals || field.widget || drawn ? 'number' : 'whole number'}${range}` + } + case 'boolean': + return 'boolean' + case 'select': { + const values = selectValues(field) + return values.length ? `${field.multiple ? 'some of' : 'one of'} ${values.map((value) => JSON.stringify(value)).join(' | ')}` : field.multiple ? 'list of values' : 'value' + } + case 'library': { + const names = [...new Set((field.options ?? []).map((/** @type {any} */ option) => String(typeof option === 'string' ? option : option?.name).split(' ')[0]))] + return `sub-content: ${names.filter((name) => !options.has || options.has(name)).join(' | ')}` + } + case 'image': + return 'image file' + case 'file': + return 'file' + case 'video': + return 'video files' + case 'audio': + return 'audio files' + default: + return field.type + } +} + +/** A group of one field is written as that field; this is the field it is written as. @param {any} field */ +function written(field) { + let shown = field + while (shown.type === 'group' && (shown.fields ?? []).length === 1 && !shown.isSubContent) { + const child = shown.fields[0] + shown = { ...child, optional: shown.optional || child.optional, label: shown.label ?? child.label, default: shown.default ?? child.default, showWhen: shown.showWhen ?? child.showWhen } + } + return shown +} + +/** + * One field's lines and, for a group or a list of groups, its children's. + * @param {any} field + * @param {string} indent + * @param {string} path the field's path, `a.b[].c` + * @param {{ left: number }} counts + * @param {SchemaOptions} options + * @param {boolean} [drawn] whether a widget of the content type's draws it, not the editor's form + * @returns {string[]} + */ +function lines(field, indent, path, counts, options, drawn = false) { + if ((options.sets ?? []).includes(path)) return [] + const hidden = field.widget === 'none' || field.widget === 'hidden' + if (hidden && field.type === 'boolean') return [] + if (!options.all && isInterfaceText(field)) { + counts.left += countTexts(field) + return [] + } + const shown = written(field) + const media = ['image', 'file', 'video', 'audio'].includes(shown.type) + const required = !shown.optional && (media || (shown.default === undefined && !['boolean', 'group', 'list'].includes(shown.type))) + let head + /** @type {any} */ + let item = null + if (shown.type === 'list') { + const range = shown.max ? ` ${shown.min ?? 0}–${shown.max}` : shown.min ? ` ${shown.min}+` : '' + item = written(shown.field ?? {}) + head = item.type === 'group' ? `list${range} of` : `list${range} of ${kind(item, options, drawn)}` + } else if (shown.type === 'group') head = 'group' + else head = kind(shown, options, drawn) + // A list the editor's own form starts with an item is required when its item needs writing; one + // a widget of a content type's draws starts empty, and needs only its minimum. + const list = shown.type === 'list' && !shown.optional && (shown.min > 0 || (!drawn && needsWriting(item))) + if (required || list) head += ', required' + if (shown.default !== undefined && shown.default !== null && shown.type !== 'group') head += ` = ${short(shown.default)}` + if (hidden) head += ', hidden in the editor' + const rules = shown.showWhen?.rules ?? [] + if (rules.length) head += `; only when ${rules.map((/** @type {any} */ rule) => `${String(rule.field).replace(/^\.\.\//, '')} is ${[rule.equals].flat().map((value) => JSON.stringify(value)).join(' or ')}`).join(' or ')}` + const label = typeof shown.label === 'string' ? plain(shown.label) : '' + const own = typeof shown.description === 'string' ? shown.description : '' + const itemDescription = item && item.type !== 'group' && typeof item.description === 'string' ? item.description : '' + const description = options.descriptions !== false && ['text', 'list', 'library', 'number', 'select'].includes(shown.type) ? plain(own || itemDescription) : '' + const about = [label, description && description !== label ? description : ''].filter(Boolean).join(': ') + const out = [`${indent}- \`${field.name}\` ${head}${about ? ` — ${about.length > 200 ? `${about.slice(0, 197)}…` : about}` : ''}`] + // What the editor shows in its "important" box: the syntax a field's text is written in. + const important = shown.important ?? (item && item.type !== 'group' ? item.important : null) + if (options.descriptions !== false && important?.description) { + const example = important.example ? ` Example: ${plain(String(important.example))}` : '' + out.push(`${indent} ${plain(String(important.description))}${example}`) + } + const inner = drawn || Boolean(shown.type === 'group' && shown.widget && shown.widget !== 'showWhen' && shown.widget !== 'none') + if (shown.type === 'group') for (const child of shown.fields ?? []) out.push(...lines(child, `${indent} `, `${path}.${child.name}`, counts, options, inner)) + if (item?.type === 'group') for (const child of item.fields ?? []) out.push(...lines(child, `${indent} `, `${path}[].${child.name}`, counts, options, inner)) + return out +} + +/** + * Whether a list's item needs something written: a required field with no default in it. + * @param {any} item + * @returns {boolean} + */ +function needsWriting(item) { + if (!item) return false + const shown = written(item) + if (shown.type === 'group') return (shown.fields ?? []).some((/** @type {any} */ child) => !child.optional && needsWriting(child)) + if (shown.optional || shown.default !== undefined || shown.type === 'boolean') return false + return shown.type !== 'list' || shown.min > 0 +} + +/** @param {any} field @returns {number} */ +function countTexts(field) { + if (field.type === 'text') return 1 + return (field.fields ?? []).reduce((/** @type {number} */ sum, /** @type {any} */ child) => sum + countTexts(child), 0) +} + +/** + * A content type's fields, one line each, nested as they are written. + * @param {any[]} semantics + * @param {SchemaOptions} [options] + * @returns {{ lines: string[], interfaceTexts: number }} + */ +export function schemaLines(semantics, options = {}) { + const counts = { left: 0 } + const out = [] + for (const field of semantics) out.push(...lines(field, '', field.name, counts, options)) + return { lines: out, interfaceTexts: counts.left } +} + +/** The line that opens a type file's generated section, and the one that closes it. */ +export const GENERATED_START = '' +export const GENERATED_END = '' + +/** + * The generated section of a type file: the fields, under a heading, between markers that a + * refresh rewrites. + * @param {any} library its `library.json` + * @param {any[]} semantics + * @param {SchemaOptions} [options] + */ +export function schemaMarkdown(library, semantics, options = {}) { + const { lines: fields, interfaceTexts } = schemaLines(semantics, options) + return [ + GENERATED_START, + '', + `## Fields of ${library.machineName} ${library.majorVersion}.${library.minorVersion}`, + '', + ...fields, + '', + interfaceTexts + ? `${interfaceTexts} interface texts are left out: they come from the libraries' translations. \`h5p-author schema ${library.machineName} --all\` lists them, for a source to set one.` + : 'It has no interface texts of its own.', + '', + GENERATED_END + ].join('\n') +} diff --git a/packages/author/lib/text.mjs b/packages/author/lib/text.mjs new file mode 100644 index 00000000..78b93ef1 --- /dev/null +++ b/packages/author/lib/text.mjs @@ -0,0 +1,159 @@ +/** + * Authored text as H5P stores it. A field without `tags` is plain text, which the platforms keep + * escaped (PHP's `htmlspecialchars`, existing entities left alone). A field with `tags` holds + * markup, as H5P's editor writes it into that field: the tags the field names, and the four the + * content check always keeps, `div`, `span`, `p` and `br`. Anything else is refused by name, so a + * source never reaches a learner in a form the platforms would have changed. + */ + +/** An `&` that does not begin an entity. */ +const BARE_AMPERSAND = /&(?!(?:[a-zA-Z][a-zA-Z0-9]*|#[0-9]+|#[xX][0-9a-fA-F]+);)/g + +/** + * Plain text as the platforms store a text field: `&`, `<`, `>`, `"` and `'` escaped, an entity + * already there kept, as `htmlspecialchars` with `ENT_QUOTES` and no double encoding keeps it. + * @param {string} value + */ +export function escapeText(value) { + return value.replace(BARE_AMPERSAND, '&').replaceAll('<', '<').replaceAll('>', '>').replaceAll('"', '"').replaceAll("'", ''') +} + +/** + * The tags a markup field keeps, as the content check derives them from its semantics: the four + * it always keeps, the field's own, and those one of them implies. `null` for a plain text field. + * @param {any} field + * @returns {Set | null} + */ +export function allowedTags(field) { + if (field.tags === undefined || field.tags === null) return null + const tags = new Set(['div', 'span', 'p', 'br', ...(Array.isArray(field.tags) ? field.tags.filter((/** @type {unknown} */ tag) => typeof tag === 'string') : [])]) + if (tags.has('table')) for (const tag of ['tr', 'td', 'th', 'colgroup', 'col', 'thead', 'tbody', 'tfoot', 'figure', 'figcaption']) tags.add(tag) + if (tags.has('b')) tags.add('strong') + if (tags.has('i')) tags.add('em') + if (tags.has('ul') || tags.has('ol')) tags.add('li') + if (tags.has('del') || tags.has('strike')) tags.add('s') + return tags +} + +/** + * The block a line of text becomes in a markup field, as H5P's editor writes it: `

` where the + * field asks for paragraphs or offers headings, `

` otherwise. + * @param {any} field + */ +export function blockTag(field) { + const own = Array.isArray(field.tags) ? field.tags : [] + return field.enterMode === 'p' || own.some((/** @type {string} */ tag) => /^(h[1-6]|pre|p|div)$/.test(tag)) ? 'p' : 'div' +} + +/** The tags a style attribute may sit on, and what it may say there, as the content check keeps them. */ +const STYLEABLE = new Set(['span', 'p', 'div', 'h1', 'h2', 'h3', 'table', 'col', 'figure', 'td', 'th', 'li']) +const COLOR = '(#[a-f0-9]{3}[a-f0-9]{3}?|rgba?\\([0-9, ]+\\)|hsla?\\([0-9,.% ]+\\))' +const BORDER_STYLE = '(none|solid|dotted|dashed|double|groove|ridge|inset|outset)' + +/** @param {any} field */ +function stylePatterns(field) { + const patterns = ['^text-align: *(center|left|right);?$'] + const font = field.font && typeof field.font === 'object' ? field.font : {} + if (font.size) patterns.push('^font-size: *[0-9.]+(em|px|%) *;?$') + if (font.family) patterns.push("^font-family: *[-a-z0-9,'&; ]+;?$") + if (font.color) patterns.push(`^color: *${COLOR} *;?$`) + if (font.background) patterns.push(`^background-color: *${COLOR} *;?$`) + if (font.spacing) patterns.push('^letter-spacing: *[0-9.]+(em|px|%) *;?$') + if (font.height) patterns.push('^line-height: *[0-9.]+(em|px|%|) *;?$') + if (Array.isArray(field.tags) && field.tags.includes('table')) { + patterns.push( + `^border: *[0-9.]+(em|px|%|) *${BORDER_STYLE} *${COLOR} *;?$`, `^border-style: *${BORDER_STYLE} *;?$`, + '^border-width: *[0-9.]+(em|px|%|) *;?$', `^border-color: *${COLOR} *;?$`, '^vertical-align: *(middle|top|bottom);?$', + '^padding: *[0-9.]+(em|px|%|) *;?$', '^width: *[0-9.]+(em|px|%|) *;?$', '^height: *[0-9.]+(em|px|%|) *;?$', + '^float: *(right|left|none) *;?$', '^border-collapse: *collapse *;?$', `^background-color: *${COLOR} *;?$` + ) + } + return patterns.map((pattern) => new RegExp(pattern, 'i')) +} + +/** A tag, a comment, or a lone `<` or `>`: the pieces the content check reads markup in. */ +const PIECE = /(<(?=[^a-zA-Z!/])||<[^>]*(>|$)|>)/g +const ATTRIBUTE = /([a-zA-Z_:][-a-zA-Z0-9_:.]*)(?:\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'>]+)))?/g +const SAFE_URL = /^(?:https?:|mailto:|ftp:|[^:]*$)/i + +/** + * Whether a value is written as markup: it has a tag. Text without one is plain, and is wrapped + * as the editor wraps what an author types. + * @param {string} value + */ +export function hasMarkup(value) { + return /<[a-zA-Z!/]/.test(value) +} + +/** + * A markup field's value as it is stored. Plain text is escaped and wrapped in the field's block, + * a paragraph per blank line and a `
` per other line break. Markup is checked: a tag the field + * does not take, an event handler, a style the field does not allow, a script URL or a comment is + * reported; a lone `<` or `>` is escaped, as the content check would escape it. + * @param {string} value + * @param {any} field + * @param {(message: string) => void} report + */ +export function markupText(value, field, report) { + const tags = allowedTags(field) ?? new Set() + if (!hasMarkup(value)) { + const block = blockTag(field) + const paragraphs = value.trim().split(/\n[ \t]*\n+/).filter((paragraph) => paragraph.trim()) + if (!paragraphs.length) return '' + return paragraphs.map((paragraph) => `<${block}>${escapeText(paragraph.trim()).replace(/\r?\n/g, '
')}`).join('') + } + const styles = stylePatterns(field) + // Each thing said once, however often the value repeats it: an opening tag and its closing one. + const said = new Set() + const once = (/** @type {string} */ message) => { + if (said.has(message)) return + said.add(message) + report(message) + } + return value.replace(PIECE, (piece) => { + if (piece === '<') return '<' + if (piece === '>') return '>' + if (piece.startsWith('', field, (message) => problems.push(message)) + expect(problems).toEqual([ + 'has an event handler, onclick, on

, which H5P removes', + expect.stringContaining('has , which this field does not take'), + expect.stringContaining('has '), + expect.stringContaining('has
'), + 'has the style "position: fixed" on , which this field does not allow', + 'has href="javascript:alert(1)": only http, https, mailto and ftp addresses are kept', + 'has a comment, which H5P removes: take it out' + ]) + }) + + it('notes a tag in a plain field, which a learner sees as written', () => { + const notes: string[] = [] + expect(plainText('What does
do?', (message) => notes.push(message))).toBe('What does <div> do?') + expect(notes).toHaveLength(1) + }) +}) + +describe('the libraries\' translations', () => { + // Names as the pack's quiz libraries have them, `sme` and Question Set's Traditional `zh` among them. + const files = ['de', 'es', 'es-mx', 'he', 'nb', 'nn', 'pt', 'pt-br', 'sme', 'sr', 'zh', 'zh-cn', 'zh-hans', 'zh-hant', 'zh-tw'] + it.each([ + ['de', ['de']], + ['de-AT', ['de']], + ['es-MX', ['es-mx', 'es']], + ['pt-BR', ['pt-br', 'pt']], + ['iw', ['he']], + ['no', ['nb']], + ['se', ['sme']], + ['sr-Cyrl-BA', ['sr']], + ['sr-Latn', []], + ['sh', []], + ['zh', ['zh-hans', 'zh-cn']], + ['zh-TW', ['zh-hant', 'zh-tw']], + ['zh-Hant', ['zh-hant', 'zh-tw']], + ['en', []] + ])('looks the translation for %s up by H5P\'s file names, never in another script', (tag, names) => { + expect(translationFiles(files, tag)).toEqual(names) + }) + + it('takes a translation\'s text defaults only, and keeps the English where it has none, the same, or loses a placeholder', () => { + const fields = [ + { name: 'check', type: 'text', default: 'Check' }, + { name: 'mode', type: 'select', default: 'auto' }, + { name: 'score', type: 'text', default: 'You got :num of :total' }, + { + name: 'texts', type: 'group', fields: [ + { name: 'question', type: 'text', default: 'Question' }, + { name: 'next', type: 'text', default: 'Next' }, + { name: 'progress', type: 'text', default: '@current of @total' }, + { name: 'results', type: 'text', default: 'Results' } + ] + }, + { name: 'items', type: 'list', field: { name: 'item', type: 'text', default: 'Item' } } + ] + const { strings, translated, english } = translateDefaults(fields, [ + { default: 'Prüfen' }, + { default: 'manuell' }, + { default: 'Du hast :num von :max' }, + { fields: [{ default: 'Question' }, { default: ' ' }, { default: '@current von @total' }] }, + { field: { default: 'Eintrag' } } + ]) + expect(strings).toBe(7) + expect(translated.map(({ field, text }) => [field.name, text])).toEqual([['check', 'Prüfen'], ['progress', '@current von @total'], ['item', 'Eintrag']]) + expect(english).toEqual([ + { field: 'score', text: 'You got :num of :total', dropped: [':total'] }, + { field: 'texts.question', text: 'Question' }, + { field: 'texts.next', text: 'Next' }, + { field: 'texts.results', text: 'Results' } + ]) + expect(translateDefaults(fields, { semantics: [] }).translated).toEqual([]) + }) +}) diff --git a/packages/author/tsconfig.json b/packages/author/tsconfig.json new file mode 100644 index 00000000..ea9cdc40 --- /dev/null +++ b/packages/author/tsconfig.json @@ -0,0 +1,17 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "ESNext", + "moduleResolution": "Bundler", + "lib": ["ES2022", "DOM", "DOM.Iterable"], + "types": ["node"], + "strict": true, + "noEmit": true, + "allowJs": true, + "isolatedModules": true, + "esModuleInterop": true, + "skipLibCheck": true, + "checkJs": true + }, + "include": ["lib/**/*.mjs", "bin/**/*.mjs", "scripts/**/*.mjs", "tests/**/*.ts"] +} diff --git a/packages/author/tsconfig.types.json b/packages/author/tsconfig.types.json new file mode 100644 index 00000000..ec74a5b8 --- /dev/null +++ b/packages/author/tsconfig.types.json @@ -0,0 +1,12 @@ +{ + "extends": "./tsconfig.json", + "compilerOptions": { + "noEmit": false, + "declaration": true, + "emitDeclarationOnly": true, + "stripInternal": true, + "rootDir": "lib", + "outDir": "types" + }, + "include": ["lib/author.mjs"] +} diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index ef8918f8..95f40a85 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -26,6 +26,9 @@ importers: specifier: ^2.0.1 version: 2.0.1 devDependencies: + '@missing-elements/h5p-author': + specifier: workspace:* + version: link:../../packages/author '@missing-elements/h5p-libraries': specifier: workspace:* version: link:../../packages/libraries @@ -56,6 +59,9 @@ importers: apps/skills: devDependencies: + '@missing-elements/h5p-author': + specifier: workspace:* + version: link:../../packages/author '@missing-elements/h5p-libraries': specifier: workspace:* version: link:../../packages/libraries @@ -78,6 +84,28 @@ importers: specifier: ^5.0.5 version: 5.0.5(@types/node@26.6.5)(@vitest/browser-playwright@5.0.5)(@vitest/ui@5.0.5)(vite@8.3.4(@types/node@26.6.5)(esbuild@0.28.2)) + packages/author: + dependencies: + '@missing-elements/h5p-libraries': + specifier: workspace:* + version: link:../libraries + '@missing-elements/h5p-normalize': + specifier: workspace:^ + version: link:../normalize + '@zip.js/zip.js': + specifier: ^2.24.0 + version: 2.24.0 + devDependencies: + '@types/node': + specifier: ^26.6.5 + version: 26.6.5 + typescript: + specifier: ^7.0.2 + version: 7.0.2 + vitest: + specifier: ^5.0.5 + version: 5.0.5(@types/node@26.6.5)(@vitest/browser-playwright@5.0.5)(@vitest/ui@5.0.5)(vite@8.3.4(@types/node@26.6.5)(esbuild@0.28.2)) + packages/cmi5: dependencies: '@missing-elements/h5p-offline-player': diff --git a/skills/h5p-author/SKILL.md b/skills/h5p-author/SKILL.md index b71d8751..55e4f9d1 100644 --- a/skills/h5p-author/SKILL.md +++ b/skills/h5p-author/SKILL.md @@ -1,153 +1,115 @@ --- name: h5p-author -description: Create an H5P quiz from a prompt, lesson, or supplied source material. Use when asked to generate, author, or build a playable .h5p Multiple Choice question or Question Set with AI. Includes a portable packaging helper, a quiz example, and the authoring-and-verification loop; no H5P server or provider API needed. +description: Create H5P content from a prompt, a lesson or supplied material, as a playable .h5p - quizzes, fill in the blanks, drag the words, flash cards, presentations, interactive books, branching scenarios, interactive video, image hotspots, memory games and more. Use when asked to generate, author or build H5P content or an .h5p file with AI. The content type is chosen with the user first; a short JSON source holds only what differs from its defaults. No H5P server or provider API needed. license: MIT --- -# Author and verify an H5P quiz - -Turn source material into a playable `.h5p`, not just JSON that looks plausible. -This first workflow supports **text-only H5P.MultiChoice and H5P.QuestionSet**. -It does not implement other content types, media, or a general H5P editor. Do not -quietly substitute a quiz if the user requested another kind of activity. - -Works with Claude Code and other agents that can read skills, write files, and run -Node commands. Ordinary chat without filesystem and command access cannot run the -workflow by itself. No AI provider SDK, API key, MCP server, or repository checkout -is required. Model calls are made by the user's agent, not by the player. -Keep [LICENSE](./LICENSE) with the installed skill and helper. Each sample's -content license is declared separately in its source JSON. - -## 1. Establish the brief - -Use what the user already supplied. Clarify missing essentials: source material, -audience, learning objective, question count, single- or multiple-answer questions, -pass mark, language, authorship/license, and destination. Do not invent factual -source material, claim expert review, or assign a license the user has not -authorized. Use `U` (undisclosed) when the license is unknown, and ask for the -author's name. - -Read [the input format](./references/quiz-format.md) and -[the worked quiz](./assets/quiz.json). Write a source JSON file in the user's -project. These strings are **plain text**, not HTML. The helper escapes them. -For a fuller example, read [the eight-scenario brief and answer key](./references/scenario-quiz.md) -with [its source JSON](./assets/phishing-scenarios.json): mixed answer modes, -explanatory feedback, an 80% threshold, and complete fail/retry/pass checks. - -For each question: - -- Test the stated objective with an answer supported by the supplied material. -- Write plausible distractors and explanatory feedback for every answer. -- Mark exactly one correct answer unless `multipleAnswers` is true. -- For multiple answers, say "Select all that apply" and mark at least one correct. -- Keep the source JSON as the editable deliverable; do not edit generated archives. - -When the quiz has a pass mark, set `passPercentage` and write `passedFeedback` and -`failedFeedback` in the content's language. They are the learner's only word on -whether they passed; the failed one should say what to do next. A Question Set does -not draw them on its result page until H5P fixes its bug, and only screen readers -announce them meanwhile (see the input format). For a Question Set, tell the user -so, since learners then need to learn the pass mark another way. - -`language` also sets the interface: the libraries' own texts (Start, Check, Retry, -Finish, the result screen) come from their translations for it, and the player's own -strings follow it. Those translations are often incomplete. A Question Set's -navigation and result screen ("Previous", "Next", "Results", "Score") are English in -all but one or two of its 48 translations. The helper lists every interface text it -left in English; tell the user which ones, rather than delivering a mixed interface -as a translated activity. -For a feature this helper does not support, explain the limit. - -## 2. Prepare the project - -Needs **Node 22 or later**. Run from a writable project directory with a -`package.json` (create one with the project's package manager if needed): - -```bash -npm install --save-dev @missing-elements/h5p-libraries@^1 @zip.js/zip.js@^2.24 +# Author H5P content + +You write a short source: the content type and only what its defaults cannot know. +`@missing-elements/h5p-author` fills in the rest from the libraries, copies them, and checks +every field. Needs Node 22 and a shell; nothing to install. Keep [LICENSE](./LICENSE) with +the skill. + +## 1. Choose the content type with the user + +Read [the brief](./references/authoring.md) once. If the user named no type, propose one or +two that fit the material and objective, a sentence each on why, and wait for their choice. + +| Type | Use it for | +|---|---| +| H5P.MultiChoice | one question, one or several right answers | +| H5P.QuestionSet | a quiz: questions in turn, then a score | +| H5P.TrueFalse | one statement, true or false | +| H5P.Blanks | typing the missing words of a text | +| H5P.DragText | dragging words into the gaps of a text | +| H5P.MarkTheWords | clicking the words of a text that fit a rule | +| H5P.SingleChoiceSet | quick questions, one right answer each | +| H5P.Summary | building a summary from true statements | +| H5P.SortParagraphs | putting paragraphs in order | +| H5P.Essay | free text, scored by keywords | +| H5P.Dialogcards | flash cards, not scored | +| H5P.Accordion | reference text in panels, not scored | +| H5P.Column | a page of text, media and questions | +| H5P.Questionnaire | a survey, no right answers | +| H5P.InteractiveBook | chapters of pages, with a summary | +| H5P.CoursePresentation | slides with text, pictures and questions | +| H5P.BranchingScenario | a story whose choices lead to different ends | +| H5P.Timeline | events along a time line | +| H5P.InteractiveVideo | questions and text over a video (needs a video) | +| H5P.ImageHotspots | a picture with spots that open text (a picture) | +| H5P.ImageHotspotQuestion | clicking the right area of a picture (a picture) | +| H5P.MultiMediaChoice | a question with pictures as answers (pictures) | +| H5P.MemoryGame | finding pairs of cards (pictures) | +| H5P.ImagePair | matching pictures to pictures (pictures) | +| H5P.ImageSequencing | putting pictures in order (pictures) | +| H5P.Dictation | hearing sentences and typing them (recordings) | +| H5P.Audio | an audio player (a recording) | +| H5P.GameMap | stages on a map, each with exercises (a map) | + +Media come from the user; never invent them. For a type not listed, run +`npx @missing-elements/h5p-author@latest schema ` and say it has no notes here. Never +read a library's semantics or code yourself. + +## 2. Write the source + +Read the type's file, `references/types/.md`: notes, an example, its fields. Parts other +content holds have files too: H5P.AdvancedText, H5P.Image, H5P.Video, H5P.Text. + +```json +{ + "type": "H5P.Blanks", "title": "Rivers", "language": "en", "license": "CC BY", + "authors": [{ "name": "Ann Author", "role": "Author" }], + "params": { "questions": ["The *Danube* flows into the Black Sea."] } +} ``` -Use the project's existing package manager if different, and keep its lockfile: -the installed library pack determines the exact library versions in the output. -If either package is already installed at an older version, the helper stops and -prints this command. -The pack contains third-party libraries under their own licenses; preserve their -files/notices and review those terms before distribution. - -Resolve `` below to **this installed skill's directory**, not a path in the -player's repository. Quote paths that contain spaces. Dependencies are resolved -from the project's current working directory, so a globally installed skill works -too. The helper reads the installed pack locally; no hub or CDN is contacted. +- `params` holds only fields that differ from their defaults. Never interface texts (Check, + Retry…): they come from the libraries' translations for `language`. +- Text is plain, or markup with only the tags a field marked `html` lists. +- A file is `{ "path": "photo.jpg" }`, beside the source or a web address, with `copyright` + when known. +- Sub-content is `{ "library": "H5P.MultiChoice", "params": { … }, "metadata": { "title": "…" } }`. +- A quiz of Multiple Choice questions alone is shorter in [the quiz format](./references/quiz-format.md), + which the build also takes. ## 3. Build ```bash -node "/scripts/build-quiz.mjs" quiz.json course.h5p +npx @missing-elements/h5p-author@latest build source.json course.h5p ``` -The helper validates the source, reads defaults from the installed libraries' -`semantics.json` and their translation for `language`, assigns each sub-question a -UUID, and writes a self-contained archive with the required runtime dependency -closure, not the entire pack. Library parts are checked against the pack index's -SHA-256 before being copied; a mismatch stops the build and names the part. Library -folders stay together before `content/`. -Existing output is never overwritten; use a new filename for a revision. +It writes nothing until the whole source is right, listing every problem by its path, and +never overwrites a file. Fix and build to a new name. Pass its notes on to the user: texts +left in English, a slide that holds too much. -There is no media in this workflow, so no normalization step is needed. If a later -workflow adds media, use `h5p-normalize` and verify the final rewritten file. - -## 4. Verify the actual output +## 4. Verify ```bash npx @missing-elements/h5p-verify@latest course.h5p -``` - -Needs Chrome/Edge or Playwright Chromium (`npx playwright install chromium`). -Keep `@latest`: an older Node can otherwise select an old verifier silently. -For a destination that plays uploads untrusted against its pack, also run: - -```bash npx @missing-elements/h5p-verify@latest course.h5p --untrusted --libraries pack ``` -Use `--libraries /path/to/parts` to check offline against a specific local pack. -Read `report.json` and open `screenshot.png` in the reported output folder. An exit -code of zero is not enough: read warnings and check the picture. The screenshot -shows startup, not every question or the final result; the preview in step 5 does. - -If `h5p-verify` is installed as a skill, follow its detailed troubleshooting loop. -Fix the source or helper, rebuild to a new output path, and verify again. Never -make verification pass by removing a question, changing the learning objective, -or hiding an error. If a browser cannot run, report the package as **unverified**. +The second plays it as a site that takes uploads untrusted. Read `report.json` and look at +`screenshot.png`: layout mistakes raise no error. Follow the `h5p-verify` skill's loop if it is +installed. Fix the source, never by removing what fails. Without a browser, say the package is +**unverified**. ## 5. Review and hand over -Check the answer key and feedback against the source, not the verifier's verdict. -Then open the quiz for the user to play in the real player, in their own browser, -adding `--untrusted --libraries pack` if the check used them: +Check answers and feedback against the material. Then let the user play it in their browser, +adding the check's `--untrusted --libraries pack` if used: ```bash npx @missing-elements/h5p-verify@latest course.h5p --preview ``` -Start it in the background; it ends about ten seconds after its tab closes. Ask the -user to answer every question, a wrong answer included, and to check the feedback, -navigation, score, retry, and the result. Tell them that the Download button beside -the file's name saves a copy of the quiz. With browser tools of your own, add -`--no-open`, play the printed address yourself, and record what you saw. The -preview is served from the machine that runs the command: if that is not the -user's computer, as in a cloud sandbox, hand over the file and the command for -them to run instead. - -- A verifier pass means startup/rendering only. Do not claim pedagogical accuracy, - accessibility conformance, full interaction testing, or compatibility with every - LMS. The pack's versions may be newer than those installed in the destination. -- Deliver the `.h5p`, source JSON, the verification report and screenshot, the - **full `revision` from the report**, and the preview command, with any remaining - warnings and limits. -- For a user who works in VS Code, Cursor or VSCodium, mention the H5P Preview - extension (`MissingElements.h5p-preview`): the quiz plays in an editor tab, and - plays again each time it is rebuilt from the source. -- To put the quiz on a website, use `h5p-player-setup`. Prefer `untrusted` with a - host-chosen pack on sites with accounts or packages from other authors. No player - page, deployment, results collection, or AI calls are added by this helper. +Run it in the background; it ends once its tab has closed. Ask the user to work through all of +it, wrong answers included; its Download button saves a copy. With browser tools of your own, +add `--no-open` and play the address yourself. Where the command does not run on the user's +computer, hand over the file and the command. + +Deliver the `.h5p`, the source, the report, screenshot and full `revision`, the preview +command, and any notes. A pass means it starts and draws, not that it is accurate, accessible +or right for every LMS. The H5P Preview extension (`MissingElements.h5p-preview`) plays it in +VS Code, Cursor or VSCodium; `h5p-player-setup` puts it on a website. diff --git a/skills/h5p-author/references/authoring.md b/skills/h5p-author/references/authoring.md new file mode 100644 index 00000000..b2f795ad --- /dev/null +++ b/skills/h5p-author/references/authoring.md @@ -0,0 +1,46 @@ +# The brief, and good content + +## What to settle with the user + +Use what the user already gave. Ask for what is missing and matters: + +- **The material**: the text, the lesson, the facts. Never invent factual material, and never + claim an expert reviewed it. +- **Who it is for, and what they should be able to do after**: one objective per activity. +- **The content type**, chosen together (SKILL.md, step 1), and its size: how many questions, + cards, slides, chapters. +- **The language**, a BCP 47 tag such as `en` or `de-AT`. It sets the libraries' interface + texts too, from their own translations, which are often incomplete. +- **Media**, for a type that needs them: files the user has, or web addresses they may use, + with who made them and under which licence. +- **Author and licence**: the author's name, and the licence for the whole, an H5P code: `U` + (undisclosed, when unknown), `CC BY`, `CC BY-SA`, `CC BY-ND`, `CC BY-NC`, `CC BY-NC-SA`, + `CC BY-NC-ND`, `CC0 1.0`, `CC PDM`, `GNU GPL`, `PD`, `ODC PDDL` or `C`; `licenseVersion` + `4.0` for a CC BY one. Never choose a licence the user has not. +- **A pass mark**, for scored content an LMS records, and what a learner who passes or fails + is told. +- **Where it goes**: an LMS, a website, a file to hand out. + +## Writing it well + +- Test the objective, with an answer the material supports. +- Wrong answers that a learner who has not understood would choose: plausible, the same length + and form as the right one. +- Feedback for every answer, wrong ones especially: why, in a sentence, not just "Wrong". +- Plain words, short sentences; one idea per question, card or slide. +- A pass mark with its messages: the failed one says what to do next. +- Alternative text for every picture that carries meaning: what it shows, for a learner who + cannot see it. +- For a quiz with a pass mark and mixed answer modes, [the eight-scenario brief](./scenario-quiz.md) + and [its source](../assets/phishing-scenarios.json) show the whole of it: answer key, + explanatory feedback, an 80% threshold, and the fail, retry and pass checks. + +## Media + +- Only files the user has or may use; a web address is downloaded into the package at build + time, so its licence travels with it. A YouTube, Vimeo or Panopto video stays there and + plays from there, which tells that site who watches. +- Pictures: `.png`, `.jpg`, `.gif`, `.bmp`. Video: `.mp4`, `.webm`. Audio: `.mp3`, `.m4a`, + `.wav`, `.ogg`. H5P takes no SVG or WebP. +- Positions on a picture (hotspots, areas, stages) are guesses until seen: look at the picture + if you can, and ask the user to check them in the preview. diff --git a/skills/h5p-author/references/quiz-format.md b/skills/h5p-author/references/quiz-format.md index b2df1b43..ebd6db94 100644 --- a/skills/h5p-author/references/quiz-format.md +++ b/skills/h5p-author/references/quiz-format.md @@ -1,8 +1,10 @@ -# Quiz source format +# The quiz format -The helper accepts this small authoring format, not raw H5P `content.json`. -Unknown fields and wrong types fail with a path; nothing is silently dropped. -All authored text is plain text. `