Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ Read it before changing the Service Worker's file server (`src/sw/mount.ts`, `pa
- Every response needs a `Content-Type`.
- `RangeHttpHandle.read` slices a `200` body as it flows, like `stream` always did.
- The archive's size never comes from a plain `GET`.
- On a host that honours `Range`, the libraries are pulled whole before the frame boots.
- On a host that honours `Range`, the libraries `h5p.json` declares are pulled whole before the frame boots.
- Media is held to a lower inline bar than everything else.
- An inline entry is inflated once per burst.
- An unknown `pkgId` is a 404 on every route.
Expand Down
Binary file modified apps/demo/demo/content/glossary.h5p
Binary file not shown.
Binary file modified apps/demo/demo/content/quiz.h5p
Binary file not shown.
Binary file modified apps/demo/demo/content/video.h5p
Binary file not shown.
5 changes: 3 additions & 2 deletions apps/demo/scripts/build-demo-content.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ import { download, hubBundle as cachedBundle } from './lib/hub.mjs'
* dependencies with it at the exact major.minor, so a Moodle with no library installed refused
* the demo's quiz, video and glossary for their editor libraries. `h5p.json` lists only what
* plays, as the platforms' exports write it, and the player never reads the rest; they follow the
* libraries that play, and the normalizer keeps library folders in that order.
* libraries that play, and the normalizer writes them after the media (`last`), so a host that
* ignores `Range` boots before they arrive.
*
* The outputs are committed. This is a maintainer's tool with two network dependencies — the
* hub, and whatever media a manifest points at — and a deploy should not have either. Hub
Expand Down Expand Up @@ -193,7 +194,7 @@ async function buildPackage(name) {

await mkdir(outDir, { recursive: true })
const output = join(outDir, `${name}.h5p`)
const report = await normalizeArchive({ input: assembled, output })
const report = await normalizeArchive({ input: assembled, output, last: [...install] })

console.log(
`[demo-content] ${name}.h5p ${formatBytes(report.bytesOut ?? 0)}: ${manifest.mainLibrary}, ` +
Expand Down
7 changes: 4 additions & 3 deletions design/author.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,9 +217,10 @@ to 3,000. Eleven things the code does not say by itself:
the selected parts over the pack's index at exact versions, through what each added library
names in turn (`installClosure`), and `writePackage` copies them after the content, where nothing
reads them as the package plays; `h5p.json` lists what plays, as the platforms' exports write it.
A package with media then goes through the normalizer, which sorts every library folder before
`content/` (`orderEntries`), so there the run precedes `content.json` and a host without `Range`
boots after it; one without media keeps the layout and boots before the run arrives. Measured on the 35
A package with media then goes through the normalizer, told to write the run after the media
(`last`, #148), so with or without media a host without `Range` boots before the run arrives;
over one with `Range` the player's warm takes only what `h5p.json` declares, and so leaves the
run out too. Measured on the 35
packages of the Moodle check, with real files: a Question Set gains 123 kB, most types under
50 kB, and the three containers much more, because their editors preload every type they can
hold: Interactive Video 2.4 → 5.1 MB, Course Presentation 1.2 → 4.8 MB (64 libraries, as
Expand Down
12 changes: 7 additions & 5 deletions design/demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,14 +323,16 @@ SPA-fallback trap described above cannot happen there.
the hub's content types not yet installed; `how-it-works.h5p` and `markup.h5p` needed nothing,
and `formulas.h5p` carries no library on purpose, so a site without Question Set refuses it, as it
does an h5p.com export). The quiz grew 123 kB, the glossary 8 kB and the video from 2.5 to
5.2 MB, an Interactive Video's editor preloading every type it can hold. The normalizer sorts
library folders before `content/`, so the added ones precede `content.json`: on a host without
`Range`, `/no-range/video.h5p` boots after 4.1 MB where it needed 1.5. The same rebuild moved the
5.2 MB, an Interactive Video's editor preloading every type it can hold. The script has the
normalizer write the added ones after the media (`last`, #148), so `content.json` stays where it
was: on a host without `Range`, `/no-range/video.h5p` boots after 1.5 MB, where the first
rebuild, with them sorted before `content/`, needed 4.1. The same rebuild moved the
quiz's Question Set, Blanks, Drag the Words, Video and Components to the hub's newer patches,
which the plan had been refreshing from the pack (the quiz now refreshes only Components, 97 to
101). A bundle has older patches than the pack for some libraries, so the video's refresh also
fetches SingleChoiceSet 1.11 (88 to 89) from the site's pack, as it would for a platform's
export: 180 kB of a library nothing plays, which the plan cannot tell from one that does. Provenance: the libraries are MIT, the text is
fetched SingleChoiceSet 1.11 (88 to 89, 180 kB) from the site's pack, a library nothing in it
plays, carried for the editor; since #148 the plan reads and refreshes only what the content
reaches, and leaves it as it is. Provenance: the libraries are MIT, the text is
ours under CC0, and the clip is Big Buck Bunny, © Blender Foundation, CC BY 3.0, credited in the
package metadata and on the player page. The `/no-range/` route serves this content on the site,
and this content or a fixture locally, which is what the browser tests need.
36 changes: 23 additions & 13 deletions design/libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,17 @@ Which libraries a package plays with: the check that it has them, the sources in
the bundle's version played, and the pack's was reported. The rule needs each own
`library.json` for the patch, so on a host with `Range` it waits for the deep index after the
warm, where they are cached (before it, names give
major and minor and nothing is refreshed). A refreshed folder is withheld from the package's own
major and minor and nothing is refreshed). Only the folders the content reaches are read, from
what it asks for at its top through each one's `preloadedDependencies`
(`PackageReader.ownLibraries`, 2026-11-07, #148): a library nothing plays, an editor library or an
older minor carried for an H5P site's upload check, keeps its name's version and is not
refreshed. The warm leaves those folders out, so reading them would be a request each, and
refreshing them downloaded parts nothing played, before the frame booted. A platform's export
carries every type its editor can hold: five from the local corpus fetched 12.7 MB of parts,
where what they play takes 1.9 (a Course Presentation 31 parts and 3.4 MB, where 3 parts and
0.33 MB play). The demo's video fetched `H5P.SingleChoiceSet` 1.11, 180 kB, and Image Sequencing
the pack's `H5P.Audio` 1.5 for the 1.2 beside its own 1.5.
A refreshed folder is withheld from the package's own
index whole (`superseded`, as `untrusted` withholds), so requests for it resolve to the part's
folder by the compatible-version rule `get()` already applies — exact for a patch, the newer
minor otherwise — and the content upgrade sees the newer version and runs its steps, as for a
Expand Down Expand Up @@ -312,15 +322,15 @@ Which libraries a package plays with: the check that it has them, the sources in
`index.js` lists what the player serves, as before, so a bundler emits only those parts, the
installable app precaches 105 and not 146, and the VS Code extension copies the served ones
with an index of them (`scripts/build.mjs`); `install.js` exports the whole pack, for a page
that writes packages with the author's browser build. What it costs the player: a package
carrying two minors of a library, Image Sequencing's `H5P.Audio` 1.2 beside 1.5, has the older
refreshed to the pack's 1.5 as any older minor is, one part downloaded that the package's own
1.5 could have answered for; accepted, one small part in a rare case, over a rule in the plan
and a refresh reported that fetched nothing. Over a host with `Range` the warm pulls the run
with the rest, as it pulls a platform export's editor libraries, one request either way
(`range-requests.test.ts` on `installable.h5p`: the probe, the tail, the head and the warm, and
the deep index then reads the run's `library.json` files from the cache); on a host without
`Range` the frame boots before the run arrives for a package without media, which keeps the
author's layout, the run after the content. A package with media goes through the normalizer,
whose `orderEntries` sorts every library folder before `content/`, and boots after the run: the
demo's video has 4.1 MB before its content, where it had 1.5.
that writes packages with the author's browser build. What it costs the player, since #148
nothing at boot: over a host with `Range` the warm takes only the folders `h5p.json` declares,
which leaves the run out as it leaves out a platform export's editor libraries (*the libraries
`h5p.json` declares are pulled whole* in serving.md), and the plan reads and refreshes only the
folders the content reaches (*The refresh rule* above), so it neither reads the run nor refreshes
an older minor in it. `range-requests.test.ts` checks that `installable.h5p`'s run is never
asked for, with the pack offered and without. Over a host without `Range` the frame boots
before the run arrives: the author writes it after the content, and a package with media goes
through the normalizer, which the author's Node build and the demo's script tell to write it
after the media (`last`). Before #148 the run sorted before `content.json` there, the warm
pulled it and the plan refreshed it: the demo's video, 5.2 MB with it, was ready at 4.5 s from
a host paced to 1 MB/s either way, where it is now 1.9 s with `Range` and 2.3 s without.
23 changes: 12 additions & 11 deletions design/normalize.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ publishes it. It is its own package, `packages/normalize`, published as
`lib/normalize.mjs` holds the policy, `lib/mp4-faststart.mjs` the remux and `lib/zip-writer.mjs`
the output side; `bin/h5p-normalize.mjs` is the command. Its documented API is `normalizeArchive`,
with `formatBytes` and `archiveFingerprint` re-exported, so the other modules stay free to change;
the demo's content build and h5p-author's Node build are the callers in this repository. Its `prepublishOnly` runs its
typecheck and tests, and its `prepack` copies the root LICENSE in. Five things about
it that the code does not say by itself:
the demo's content build and h5p-author's Node build are the callers in this repository. Its
`prepublishOnly` runs its typecheck and tests, and its `prepack` copies the root LICENSE in. Five
things about it that the code does not say by itself:

- **It writes the zip itself.** zip.js reads the input — descriptors, zip64, inflate, all
handled — but its `ZipWriter`, asked not to write a data descriptor, buffers the whole entry
Expand All @@ -30,14 +30,15 @@ it that the code does not say by itself:
write after the media, `last` (2026-11-06, #148): libraries a package carries and nothing reads
as it plays, which h5p-author and the demo's packages carry for an H5P site's upload check
(#146) and which sorted before `content.json` like any library, so that a host that ignores
`Range` had them to wait for. Only the caller knows which they are, so the command has no flag
and the normalizer reads no `library.json`; `content` is never moved and a name the archive
lacks is ignored. Measured on the demo's video with them (Interactive Video, 5.2 MB, its content
ending at 4.1 MB in the library order and at 1.5 with `last`) from a host paced to 1 MB/s that
ignores `Range`: ready at 2.3 s with 2.3 MB sent, where it was 4.5 s with 4.4 MB. From a host
that honours `Range` the boot is the same, 4.5 s either way: the warm pulls every text run
before `ready`, and the trailing run is a span of its own, eight requests for seven. Writing
the run last does not shorten that; trimming the warm would, and is the player's.
`Range` had them to wait for. h5p-author's Node build and the demo's content script pass them.
Only the caller knows which they are, so the command has no flag and the normalizer reads no
`library.json`; `content` is never moved and a name the archive lacks is ignored. Measured on
the demo's video with them (Interactive Video, 5.2 MB, its content ending at 4.1 MB in the
library order and at 1.5 with `last`) from a host paced to 1 MB/s that ignores `Range`: ready
at 2.3 s with 2.3 MB sent, where it was 4.5 s with 4.4 MB. A host that honours `Range` gains
nothing from the order: the player's warm leaves out what `h5p.json` does not declare,
wherever it lies (*the libraries `h5p.json` declares are pulled whole* in serving.md), and the
same video is ready there at 1.9 s in either layout.
- **It drops what the player would drop.** Directory entries, names that are not plain relative
paths, duplicates after normalisation — read with `filenameValidation: 'tolerant'` for the same
reason the player uses it. `plainRelativeName` is a copy of the player's `normalizeEntryName`,
Expand Down
23 changes: 22 additions & 1 deletion design/serving.md
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ How a package's bytes reach the H5P runtime: the Service Worker's virtual file s
applied to ranges too when the request accepts it; `tests/browser/compressing-host.test.ts`
pins the browser behaviour the probe depends on, and `tests/unit/source-probe.test.ts` covers
the fallback with CORS-filtered headers.
- **On a host that honours `Range`, the libraries are pulled whole before the frame boots.** The
- **On a host that honours `Range`, the libraries `h5p.json` declares are pulled whole before the frame boots.** The
runtime reads them exhaustively at boot — every `library.json`, script and stylesheet — and an
inline entry read on demand cost two ranged requests, the 30-byte local header and then the
data (one since the block cache, or none when its block is already held). Measured against a real host (nginx in front of a store, `stream.dev.mkis.fwu.de`): a
Expand All @@ -209,6 +209,27 @@ How a package's bytes reach the H5P runtime: the Service Worker's virtual file s
a picked file and a downloaded archive are local reads. `tests/unit/warm.test.ts` builds a real
archive with a video between two library folders and checks the split, the walk and the marker;
`tests/browser/warm.test.ts` checks the phase, the cache and the skip on a second load.

Only the library folders `h5p.json` declares are candidates, beside the root and `content/`
(`declaredFolders`, 2026-11-07, #148). An export lists there every library its content plays,
flattened, and none it carries only for an H5P site's upload check: the editor libraries, and an
older minor one of them names, which h5p-author's packages carry too since #146. Until then the
warm pulled them with the rest. Across the 88 real packages of the local corpus, 70 carry such
folders, and the warm came to 231 MB against 149 MB for what plays; a platform's Course
Presentation pulled 5.3 MB where its content needs 1.8. In all 88, `h5p.json` declared every
folder the dependency walk reaches, so nothing has to be read to decide; a folder left out is
served on demand if anything asks for it, as every file was before the warm, and an `h5p.json`
that declares nothing the archive carries keeps every folder in. Measured from a host paced to
1 MB/s: the demo's video ready at 1.9 s where it took 4.5 (1.9 before #146 added its editors),
that Course Presentation at 2.2 s where it took 5.8, an Interactive Video export at 2.5 s where it
took 4.4. Leaving a folder out between two that play can split a run, a request more each: the
Interactive Video's load went from five requests to nine, a Column export's from eight to
thirteen, and with 100 ms added to every request and the link at 10 MB/s the two were still
ready sooner, 1.4 s for 1.6 and 1.7 s for 2.5.
The pack's plan reads only the folders the content reaches, for the same reason: what the warm
leaves out would otherwise be read one request at a time (*The refresh rule* in libraries.md).
`tests/browser/range-requests.test.ts` checks that `installable.h5p`'s run is never asked for,
with the pack offered and without.
- **Media is held to a lower inline bar than everything else.** `INLINE_MAX_SIZE` is 16 MB, and an
inline entry is copied whole before its first byte is served; a media element wants the head
first and the rest by ranges. So `chooseStrategy` puts media over `MEDIA_INLINE_MAX_SIZE`
Expand Down
2 changes: 2 additions & 0 deletions docs/libraries.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ the fixes and the look of current H5P.
- A library the pack does not have keeps the package's own copy, and runs against the pack's
newer versions of the libraries it shares: H5P's rule is that a minor does not break what
depends on it.
- A library the package carries that its content never plays, an editor library or an older
minor kept for an H5P site's upload check, stays as it is, and nothing is downloaded for it.

Content written for an older minor is upgraded before it starts: the player runs the steps each
library ships in its `upgrades.js` on the content's parameters, as the H5P plugins do: a Question
Expand Down
5 changes: 2 additions & 3 deletions packages/author/lib/build.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -225,9 +225,8 @@ export async function composePackage(source, pack, media) {
* What a package holds but its media, in the order a package streams in: its manifest, the
* libraries it plays with, each part's own files copied as they are, the content, then the
* libraries an H5P site's upload check wants, which nothing reads as the package plays, so the
* frame boots before they arrive on a host without `Range`, unless the normalizer, which a package
* with media goes through, sorts library folders before the content. The caller adds the media
* after it, last.
* frame boots before they arrive on a host without `Range`. The caller adds the media after it,
* last, and in Node has the normalizer keep those libraries after the media (`last`).
* @param {import('@zip.js/zip.js').ZipWriter<unknown>} writer
* @param {{ manifest: Record<string, unknown>, content: Record<string, unknown>, selected: Map<string, import('./pack.mjs').PartLibrary>, install: Map<string, import('./pack.mjs').PartLibrary> }} built
* @param {import('./pack.mjs').Pack} pack
Expand Down
4 changes: 2 additions & 2 deletions packages/author/lib/install.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@
* A fresh site has nothing, so a build carries the closure (#146): the editor libraries, and the
* older minor a library names where the pack plays a newer one. The pack carries them as parts
* the player never serves (`installOnly` in its index), and a build writes them after the
* content, where nothing reads them as the package plays; the normalizer, which a package with
* media goes through, sorts library folders before `content/`.
* content, where nothing reads them as the package plays, and in a package with media after the
* media too: the Node build names them to the normalizer as `last`.
*/

/** @param {{ machineName: string, majorVersion: number | string, minorVersion: number | string }} library */
Expand Down
6 changes: 4 additions & 2 deletions packages/author/lib/node.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -143,9 +143,11 @@ export async function buildPackage(input, output, options = {}) {
}
let final = staged
if (media.files.length) {
// Media stored and last, an mp4's index first: the layout a package streams in.
// Media stored and last, an mp4's index first: the layout a package streams in. The
// libraries carried for an H5P site's upload check go after the media, where `writePackage`
// put them after the content: a host that ignores `Range` then boots before they arrive.
final = join(temporary, 'normalized.h5p')
await normalizeArchive({ input: staged, output: final })
await normalizeArchive({ input: staged, output: final, last: [...built.install.keys()] })
}
// Unlike a hard link, an exclusive copy also works on FAT/exFAT destinations.
await copyFile(final, output, constants.COPYFILE_EXCL).catch((error) => {
Expand Down
Loading
Loading