Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
153 changes: 50 additions & 103 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,19 @@
# AGENTS.md

Working notes for this repository. This file is the index: what the project is, how to work in
it, and which file holds each area's rules. [`design/`](design/) holds the rules in full, with the
reasons and the measurements behind them, one file per area; together they are the authority on
the design, and where they and the code disagree, one of them is wrong. **Before changing code in
an area, read its file in `design/`.** A name in italics, *like this*, is a rule or a section
there: grep `design/` for it. The README and `docs/` are written for people integrating the
packages.
The index for working in this repository. [`design/`](design/) holds the rules in full, one file
per area, with the reasons and the measurements behind them; it is the authority on the design, and
where it and the code disagree, one of them is wrong. **Before changing code in an area, read its
file in `design/`:** its rules are the bold sentences there, and its opening says what it
covers. A name in italics, *like this*, is a rule or a section there: grep `design/` for it. The
README and `docs/` are written for people integrating the packages.

## What this is

Self-hosted H5P without an H5P server: a browser-only player, shipped as one web component,
that a host puts on infrastructure it controls. That is the product's main benefit, and the
documents lead with it. It plays an arbitrary `.h5p` archive
fetched from a URL or picked from disk, with no server-side code and no extraction step: the zip
is read in place and a Service Worker serves its entries to the H5P runtime over a virtual file
server.
Self-hosted H5P without an H5P server: a browser-only player, shipped as one web component, that a
host puts on infrastructure it controls. That is the product's main benefit, and the documents lead
with it. It plays an arbitrary `.h5p` archive fetched from a URL or picked from disk, with no
server-side code and no extraction step: the zip is read in place and a Service Worker serves its
entries to the H5P runtime over a virtual file server.

The hard constraints that shape every file here:

Expand All @@ -30,41 +28,50 @@ The hard constraints that shape every file here:

## The three contexts

Most bugs here come from forgetting which context a line runs in.
Most bugs here come from forgetting which context a line runs in. The Service Worker is stateless
by assumption, and asks a page for what only a page can do through the frame:
[`design/serving.md`](design/serving.md).

| Context | Lives as long as | Can | Cannot |
|---|---|---|---|
| Page (element) | The tab | Hold the `File`, own the Jobs worker, emit DOM events | Touch package bytes |
| Service Worker | One event, unpredictably | Serve requests, read IndexedDB and the chunk store | Run long work, spawn a Worker, hold state across events |
| Jobs worker | The tab | Download, inflate, take Web Locks | Intercept requests, reach the DOM |

The Service Worker is **stateless by assumption**. Any handler may be the first one after a
restart, so it rebuilds readers from the `packages` table on demand. When it needs something only
a page can do, it messages the frame client (`event.clientId`), the frame relays to its parent
element, and the element acts. That relay is why `frame-document.ts` has a `message` listener.

## Where things are

A pnpm workspace of eleven parts: nine packages and two apps. Eight of the packages are published
to npm; the ninth, the VS Code extension, is a `.vsix`. In `design/`, `src/…` and `scripts/…` mean
the player's unless a file says otherwise. [`design/layout.md`](design/layout.md) maps every file.

| Part | What it is |
|---|---|
| `packages/player/` | `@missing-elements/h5p-offline-player`, MIT: the `<h5p-player>` element, the Service Worker (`src/sw/`), the Jobs worker (`src/jobs/`), what the three contexts share (`src/shared/`), and `./plan`, its library rules for author and embed |
| `packages/runtime/` | `@missing-elements/h5p-runtime`, GPL-3.0: the H5P core vendored at a commit, and the boot script that runs in the frame |
| `packages/libraries/` | `@missing-elements/h5p-libraries`: the library pack, one archive per library, checked against the H5P hub every week |
| `packages/normalize/` | `@missing-elements/h5p-normalize`: rewrites a package so it streams |
| `packages/verify/` | `@missing-elements/h5p-verify`: plays a package headless and reports, or opens it for a person |
| `packages/author/` | `@missing-elements/h5p-author`: a playable package of any content type from a short source |
| `packages/cmi5/` | `@missing-elements/h5p-cmi5`: a page holding the element as a cmi5 assignable unit |
| `packages/embed/` | `@missing-elements/h5p-embed`: the embed site for a player domain, and a package as a SCORM zip |
| `packages/vscode/` | H5P Preview, the VS Code extension |
| `apps/demo/` | the dev server for the whole repository, and the hosted site and its installable app |
| `apps/skills/` | the agent skills' test project |
| `skills/` | the agent skills, at the root, where `npx skills add` looks |
| `docs/` | the README's longer sections, for integrators |
| `design/` | the design rules in full |
## Where things are, and whose rules

A pnpm workspace of nine packages and two apps. [`design/layout.md`](design/layout.md) maps every
file; in `design/`, `src/…` and `scripts/…` mean the player's unless a file says otherwise.

| Part | What it is | Rules |
|---|---|---|
| `packages/player/` | the `<h5p-player>` element, MIT: the Service Worker in `src/sw/`, the Jobs worker in `src/jobs/`, what they share in `src/shared/` | by area, below |
| `packages/runtime/` | the H5P core, vendored, and the frame's boot script, GPL-3.0 | [frame](design/frame.md) |
| `packages/libraries/` | the library pack, one archive per library | [libraries](design/libraries.md) |
| `packages/normalize/` | rewrites a package so it streams | [normalize](design/normalize.md) |
| `packages/verify/` | plays a package headless and reports | [verify](design/verify.md) |
| `packages/author/`, `skills/h5p-author/` | a playable package of any content type from a short source | [author](design/author.md) |
| `packages/cmi5/` | the element as a cmi5 assignable unit | [cmi5](design/cmi5.md) |
| `packages/embed/`, `skills/h5p-to-scorm/` | the embed site for a player domain, and a package as a SCORM zip | [embed](design/embed.md), [scorm](design/scorm.md) |
| `packages/vscode/` | H5P Preview, the VS Code extension | [vscode](design/vscode.md) |
| `apps/demo/`, `vercel.json`, `api/` | the dev server for the whole repository, the hosted site and its app | [demo](design/demo.md) |
| `apps/skills/`, `skills/` | the agent skills, and the project they are tested from | [testing](design/testing.md) |

The player, by area:

- [Serving archives](design/serving.md): the Service Worker's file server, how a source is read, the warm.
- [Storage and the Jobs worker](design/storage.md): the chunk store, eviction and quota, the Jobs worker.
- [Libraries](design/libraries.md): which libraries a package plays with, the pack, refresh, addons.
- [The frame and the runtime](design/frame.md): the frame document, the boot route, the runtime package.
- [Results and saved state](design/results.md): statements and `revision`, `finished`, `resume`.
- [The element](design/element.md): loads, the worker's registration and upgrade, the events, the box.
- [Untrusted packages and security](design/security.md): entry names, `untrusted`, the content check, the frame's policy and messages.

Across the repository: build shape, types, publishing and Node versions (22 LTS or 24 LTS) in
[`design/build.md`](design/build.md); tests and CI in [`design/testing.md`](design/testing.md);
licensing and names in [`design/licensing.md`](design/licensing.md), where the player is MIT and
carries no GPL code, reaching `@missing-elements/h5p-runtime` only over HTTP and `postMessage`; what
is not built yet in [`design/scope.md`](design/scope.md).

## Commands

Expand All @@ -84,67 +91,7 @@ pnpm verify course.h5p # play a package in a headless browser and re
pnpm author build source.json course.h5p # after pnpm build, or the player's build:plan
```

Every other command (the demo's content, images and screenshots, the library pack, translations,
the core's sync, the sweep, the VS Code extension) and what to know when running them, such as why
`pnpm test` runs one package at a time: [`design/commands.md`](design/commands.md).

Every other command, and what to know when running them: [`design/commands.md`](design/commands.md).
**What is tested**, by the owner's rule: the packages and their source, and of the demo app only
the installable app; the demo's pages are checked by hand, and the skills are tested from
`apps/skills`: [`design/testing.md`](design/testing.md). Node 22 LTS or 24 LTS:
[`design/build.md`](design/build.md).

## Design rules, by area

Read the area's file before changing code in it: its rules are the bold sentences there.

- **Serving archives**, [`design/serving.md`](design/serving.md): before changing the Service
Worker's file server (`src/sw/mount.ts`, `package-reader.ts`, `routes.ts`, `watermark-wait.ts`),
how a source is read (`src/shared/source.ts`, `block-cache.ts`, `source-reader.ts`,
`span-stream.ts`, `resilient-stream.ts`, `range.ts`, `local-header.ts`, `forward-index.ts`,
`mime.ts`) or the warm (`src/jobs/warm.ts`).
- **Storage and the Jobs worker**, [`design/storage.md`](design/storage.md): before changing the
chunk store, eviction or quota (`src/shared/chunk-store.ts`, `eviction.ts`, `locks.ts`,
`storage.ts`, `idb.ts`) or the Jobs worker (`src/jobs/`).
- **Libraries**, [`design/libraries.md`](design/libraries.md): before changing which libraries a
package plays with: the library check, the `libraries` sources, the pack and its parts, refresh,
downgrade, addons, the merged manifest (`src/sw/pack-plan.ts`, `src/shared/library-folders.ts`,
`content-libraries.ts`, `pack.ts`, `src/plan.ts`, the element's `index()`, `packages/libraries/`).
- **The frame and the runtime**, [`design/frame.md`](design/frame.md): before changing the frame
document or the boot route (`src/sw/frame-document.ts`, `library-load.ts`) or the runtime package
(`packages/runtime/`: the boot script, upgrades, translations).
- **Results and saved state**, [`design/results.md`](design/results.md): before changing what
reaches the host: statements and `revision`, `finished`, `resume` (`src/shared/revision.ts`,
`src/user-data-store.ts`, `packages/runtime/src/finished.ts`).
- **The element**, [`design/element.md`](design/element.md): before changing
`src/h5p-offline-player.ts`: loads, the worker's registration and upgrade, the events, the
element's box and its height.
- **Untrusted packages and security**, [`design/security.md`](design/security.md): before changing
entry names, `untrusted`, the content check (`packages/runtime/src/content-check.ts`), the frame's
policy or the messages between the frame and the element.

## The other packages, the site and the rest

- **The normalizer** (`packages/normalize/`): [`design/normalize.md`](design/normalize.md).
- **The verifier** (`packages/verify/`): [`design/verify.md`](design/verify.md).
- **The author package** (`packages/author/`, `skills/h5p-author/`):
[`design/author.md`](design/author.md).
- **The VS Code extension** (`packages/vscode/`): [`design/vscode.md`](design/vscode.md).
- **The SCORM package** (`packages/embed/lib/scorm*.mjs`, `packages/embed/scorm/`, the SCORM page,
`skills/h5p-to-scorm/`): [`design/scorm.md`](design/scorm.md).
- **The embed site** (`packages/embed/` but its SCORM side, the demo's `/embed`):
[`design/embed.md`](design/embed.md).
- **cmi5** (`packages/cmi5/`, the demo's cmi5 page, CATAPULT): [`design/cmi5.md`](design/cmi5.md).
- **The hosted demo and the app** (`apps/demo/`, `vercel.json`, `api/`):
[`design/demo.md`](design/demo.md). It compiles the element from the player's source, not from
`dist/`, as [`design/layout.md`](design/layout.md) explains.
- **Build shape, types and publishing** (the artefacts and how each is built, the plain-JS packages'
declarations, Node versions): [`design/build.md`](design/build.md).
- **Tests and CI** (the engines, CI's browsers, the fixtures, the sweep):
[`design/testing.md`](design/testing.md).
- **Licensing and names**: [`design/licensing.md`](design/licensing.md). The player is MIT and
carries no GPL code: the H5P core and the boot script are `@missing-elements/h5p-runtime`,
GPL-3.0, reached only over HTTP and `postMessage`, and every shipped file with third-party code
opens with its notice. A name that leads with "H5P" says who made it ("… by Missing Elements"), as
H5P Group's trademark policy asks.
- **Scope** (what differs from the original architecture document, and what is not built yet):
[`design/scope.md`](design/scope.md).
`apps/skills`.
62 changes: 61 additions & 1 deletion design/author.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ already holds text to the content check's rules, from the pack an untrusted chec
`apps/skills` plays every example untrusted too. Measured on a five-question quiz, at four
characters a token: `content.json` was about 3,470 tokens to write by hand, after reading 3,900 to
9,500 of a type's semantics; the source is 1,000 to 1,800 tokens, written from a type file of 200
to 3,000. Thirty things the code does not say by itself:
to 3,000. Thirty-two things the code does not say by itself:

- **One generator, three ways to run it.** The third, `saved`, checks content as H5P stores it
(*Saved content is checked as the build checks what it writes*, below). `ContentGenerator` was the sweep's
Expand Down Expand Up @@ -111,6 +111,66 @@ to 3,000. Thirty things the code does not say by itself:
normalizer's `normalizeArchive`, so it is stored and laid out to stream, an mp4's index first.
An addon whose pattern the text matches, H5P.MathDisplay for LaTeX, is carried as a site with
it installed exports it.
- **A build that acts on what a model wrote can be confined** (2026-11-17, #220, for the local MCP
server of #219). The command line reads any path a source names and fetches any address, which is
right for a person's own command. `buildPackage` takes `within`, a list of the folders the output
and every local file a source names must be inside, judged on real paths, so `..` and a link out
are refused; one folder given alone, as a string, is refused, since it would be read letter by
letter, `/` among them, and confine nothing. It is judged before the file is looked for, and a
confined build's problems name no path but the one written: the first cut said "outside" of a
file that was there and "no such file" of one that was not, so a model could map the disk around
the folders, and it named every absolute path, the user's name with it. A file the build cannot
read is a problem of the source's as well, found where the file is looked for, and not a failure
halfway through that names its path. `check` builds and writes no package (`build --check`),
running the same check as a build, `checkContent` being `composePackage`'s, and takes no output;
a media file named by web address is still downloaded, to a temporary folder removed after, as
the check reads its bytes, and a video on YouTube, Vimeo, Panopto or Echo360 stays an address, as
in a build. `publicFetch`, passed as `fetch`, refuses every network that is not the
public internet on every hop, through `net.BlockList`, which matches an IPv4 address however IPv6
writes it: the first cut checked dotted text, and a URL writes `[::ffff:127.0.0.1]` as
`[::ffff:7f00:1]`, which reached a server on loopback. The local IPv4 ranges are refused behind
NAT64's prefix and in 6to4 too, which a gateway carries to an IPv4 network, and with them the IPv6
ranges no web host is on: IPv4-translated, Teredo, benchmarking, documentation, discard-only and
segment routing's. A name that does not resolve is refused in the words a local one is, so a
refusal does not tell which names a local network has. A redirect to another origin drops
`Authorization`, `Proxy-Authorization` and `Cookie`, as `fetch` does, and the signal stops a
lookup, which takes none of its own. It resolves a name and then asks, so it guards against a
model's mistake or a planted address, not a hostile network. Nor is `within` a guard against
another process racing on the folders, swapping a path for a link between its check and its use:
one that can write there already acts as the user, and Node has no directory handles to hold a
path while it is used.
- **The outline is the content for a review, read through the semantics** (2026-11-17, #220). Every
result has `outline`, all of it, made when it is first read, so a build that never reads it never
runs it. `outlineOf(libraries, content)` takes its arguments as `checkContent` does, refuses them
the other way round rather than answer with nothing, and cuts at a `limit` only where one is
given. It writes the text fields a learner reads that hold more than their default, which in
loaded semantics is the content language's, so interface texts drop out in any language; a rich
text's blocks and line breaks apart, by ` / `, and a tag inside a word taken out with nothing in
its place (`H<sub>2</sub>O`); a ✓ on a `correct` item, a `correct` select's answer, a score range
before its feedback in a RangeList. Settings are left out, and with them a group or a list the
editor does not draw (Game Map's stage positions) and a text no label names, which a widget of the
type's own sets (Find the Hotspot's figure, Virtual Tour's positions): by a scan of every library
in the pack, the only unlabelled texts but a list's items and those `TYPES` labels. What a few
types say only by position or number is that one table, `TYPES`: Single Choice Set's and
Summary's first item is the right one, as their semantics say in words alone; slides, chapters,
questions and a scenario's content are numbered; a question's title the build gave
(`questionTitle`, the set's own label and the number, shared with its layout) is not written; a
scenario's links name those numbers, a question's own link left out, since its alternatives lead;
an Interactive Video's interactions and bookmarks start with their times, a bookmark named as
one. The first cut read keys, not semantics: it dropped every number ("1969" as an answer), hid
True/False's answer, gave a Question Set 29 lines of which 23 were interface texts, ran a heading
into its paragraph, and showed no slide, link or time. The tags a text holds are taken out by
`stripTags`, one function in `text.mjs` for the generator, the layouts, `schema` and the outline,
which reads markup in the content check's pieces (`PIECE`, as `markupText` checks a field): a tag
runs from a `<` before a letter, `/` or `!` to the first `>`, and each is a space, never nothing,
which is what CodeQL asks of a tag remover (alert #46 on the first cut). A second cut read quotes
in a tag as a browser does, and took the apostrophe of an unquoted attribute (`href=it's`) for the
start of a quoted one, running a link's tag into the text after it. Read as the check reads it, a
`<` inside an attribute no longer cuts a tag short and a `<` that starts none is text: the one
change to what a build writes without the options, its result having `outline` besides, where the
generator's check for an empty field and the layouts' measures had read part of a tag as text, or
a text as a tag. The outline splits rich text at its tags (`tagsIn`) and plain text at its line
breaks before it reads entities, so no text, an encoded one included, can pass for a break.
- **The pack is pinned.** The dependency is `workspace:*`, published as the exact version
packages/libraries has, so a release builds with the libraries it was tested with and one command
needs no project set up; `--libraries` names another `parts/` folder. Each part is checked
Expand Down
Loading
Loading