Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion design/author.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ already holds text to the content check's rules, from the pack an untrusted chec
`apps/skills` plays every example untrusted too. Measured on a five-question quiz, at four
characters a token: `content.json` was about 3,470 tokens to write by hand, after reading 3,900 to
9,500 of a type's semantics; the source is 1,000 to 1,800 tokens, written from a type file of 200
to 3,000. Twelve things the code does not say by itself:
to 3,000. Thirteen things the code does not say by itself:

- **One generator, three ways to run it.** The third, `saved`, checks content as H5P stores it
(*Saved content is checked as the build checks what it writes*, below). `ContentGenerator` was the sweep's
Expand Down Expand Up @@ -277,6 +277,27 @@ to 3,000. Twelve things the code does not say by itself:
an authoring tool's public address without "H5P" in it, and that origin already serves the
player and the pack. It builds from `@missing-elements/h5p-libraries/install` at the version
the author package pins, so the page plays with the pack it built from.
- **A bundle per content type, where the pack cannot be installed** (2026-11-09, #166). The page
leaves the last steps to the user, but ChatGPT's sandbox runs Node and fetches a file its
browsing tool has opened: given the libraries as files, a chat builds and fixes its source
itself. The skill chooses the content type first, so the unit is the type. `bundleLibraries` is
the build's own rules over the most a source of the type can name: the type and every library
its library fields offer or give as their default, theirs in turn; `planPack` over all of them
as roots, as `closure` in `build.mjs` plans one package; `installClosure` over what that plans.
Both only ever add for more roots, so one source's libraries are within its type's bundle.
`writeBundle` zips them as `parts/`, with `libraries.txt`, without which the parts are not handed
on. The bundle carries the pack's whole index, not its own entries: a build from it plans as it
would from the pack, so it writes the same package (the skill's 32 examples were built both ways
and matched, but for the ids a build draws) or stops and names the part it lacks, where a cut-down
index would build a source with math without its addon and say nothing. No addon is in a
bundle: H5P.MathDisplay, 2.8 MB, joins wherever text holds math, and would more than double most
bundles, so a source that calls for it fetches that one part. A bundle rather than the parts,
since ChatGPT's download tool costs two calls a file, opened and then fetched: a Question Set's
33 parts would be 66. With pack 1.1.0 a question type's is 0.6 to 1.5 MB, a Question Set's
2.6 MB, a container's 4.3 to 8.4 MB, and the 55 content types' 99 MB in all, which is why
embed-my.github.io serves them (embed-my/embed-my.github.io#14) rather than the libraries
package on jsDelivr, whose cap is 50 MB. The build tool beside them is the command line as one
file, esbuild's: 558 kB, and it builds from an unzipped bundle with nothing installed.

Not built: editing an existing package, which a tool beside it does with `checkContent` as its
check; layout for Drag and Drop and Crossword; and any judgement of whether content is right, which
Expand Down
27 changes: 25 additions & 2 deletions packages/author/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,26 @@ H5P.TrueFalse 1.8: True/False Question
`--all` adds the interface texts; `--markdown` prints the section the skill's type files carry.
An unknown name lists the pack's content types.

## A content type's bundle

```bash
npx @missing-elements/h5p-author@latest bundle H5P.QuestionSet H5P.QuestionSet.zip
```

```
Bundled H5P.QuestionSet 1.21 in /work/H5P.QuestionSet.zip: 33 parts from the library pack 1.1.0, 2.6 MB. Unzip it beside a source and build with --libraries parts.
```

The libraries a build of one content type reads, for a build where the pack cannot be installed,
such as a chat's sandbox that reaches no npm registry: everything the type can hold, with the
editor libraries an H5P site's upload check asks for. The zip holds `parts/`, the pack's whole
index with those parts, and `libraries.txt`, the licences they are used under. A build from it
writes the package the pack would. An addon is not in it: where a source's text calls for one,
H5P.MathDisplay for math, the build stops and names the missing part, to copy into `parts/` from
the pack's own `parts/` folder. With pack 1.1.0 a question type's bundle is about 1 MB, a Question
Set's 2.6 MB, a container's up to 8.4 MB. `--all <folder>` writes one per content type,
`<machine name>.zip`: 55, 99 MB in all. If one fails, none is left behind.

## Options

- `--libraries <folder>` or `--libraries=<folder>`: another copy of the pack's `parts/` folder, in
Expand All @@ -128,8 +148,11 @@ const { output, manifest, content, libraries, install, translations, notes, medi
plays with, `install` those carried for an H5P site's upload check. `missingForInstall(libraries)`
is that check, over a package's `library.json` files: what a site with no library installed would
refuse it for, empty for a build. `schemaOf(machineName, { markdown, all,
libraries })` returns the field list. `ContentGenerator` is the engine underneath, also run by the
repository's sweep of every content type.
libraries })` returns the field list. `writeBundle(type, output, { libraries })` and
`writeBundles(folder, { libraries })` write bundles, each `{ output, type, parts, size }`;
`bundleLibraries(pack, type)` returns a bundle's parts, and `contentTypes(pack)` the pack's content
types. `ContentGenerator` is the engine underneath, also run by the repository's sweep of every
content type.

A tool beside the build takes the libraries as the build loads them, the generator, and the
build's rules for text:
Expand Down
25 changes: 24 additions & 1 deletion packages/author/bin/h5p-author.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,23 @@
import { readFile } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { parseArgs } from 'node:util'
import { buildPackage, schemaOf, SourceError } from '../lib/author.mjs'
import { buildPackage, schemaOf, SourceError, writeBundle, writeBundles } from '../lib/author.mjs'
import { packVersion } from '../lib/node.mjs'

const USAGE = `Usage:
h5p-author build <source.json> <output.h5p> [--libraries <parts folder>]
h5p-author schema <machine name> [--markdown] [--all] [--libraries <parts folder>]
h5p-author bundle <machine name> <output.zip> [--libraries <parts folder>]
h5p-author bundle --all <folder> [--libraries <parts folder>]

build Makes a playable .h5p from a source: { "type", "title", "language", "license",
"authors", "params" }, or a quiz in the h5p-author skill's quiz format. Writes only
when the whole source is right, and never over an existing file.
schema Prints a content type's fields as a source writes them: --markdown as a skill's
type file carries them, --all with the interface texts.
bundle Writes the libraries a build of one content type reads, as a zip of a parts/
folder for build --libraries parts, where the pack cannot be installed; --all,
one per content type, named by it.

-h, --help show this
-v, --version print this package's version and the pack's
Expand All @@ -22,6 +27,9 @@ The libraries come from the pack this release pins (@missing-elements/h5p-librar
--libraries names another copy's parts/ folder. Check every package with
npx @missing-elements/h5p-verify@latest before handing it over.`

/** A size in kilobytes below a megabyte, in megabytes above. @param {number} bytes */
const sized = (bytes) => (bytes < 1_000_000 ? `${Math.max(1, Math.round(bytes / 1000))} kB` : `${(bytes / 1_000_000).toFixed(1)} MB`)

/**
* The command line as the verifier's and embed's read theirs: `--libraries dir` or
* `--libraries=dir`, and `-h`, `-v`. Which flags a command takes is checked after.
Expand Down Expand Up @@ -85,6 +93,21 @@ try {
}
for (const { path, message } of result.notes) console.warn(`Note: ${path} ${message}`)
console.log('Not verified: run npx @missing-elements/h5p-verify@latest on it and look at its screenshot before handing it over.')
} else if (command === 'bundle') {
for (const flag of Object.keys(flags)) if (flag !== 'all' && flag !== 'libraries') throw new Error(`bundle does not take --${flag}`)
const libraries = /** @type {string | undefined} */ (flags.libraries)
const from = libraries ? `the pack in ${libraries}` : `the library pack ${await packVersion()}`
if (flags.all) {
if (rest.length !== 1) throw new Error('bundle --all takes the folder to write them in: h5p-author bundle --all bundles')
const results = await writeBundles(rest[0], { libraries })
for (const { type, parts, size } of results) console.log(`${type}: ${parts.length} parts, ${sized(size)}`)
const total = results.reduce((sum, { size }) => sum + size, 0)
console.log(`Bundled ${results.length} content types in ${resolve(rest[0])} from ${from}, ${sized(total)} in all.`)
} else {
if (rest.length !== 2) throw new Error('bundle takes a content type and an output: h5p-author bundle H5P.QuestionSet H5P.QuestionSet.zip')
const { output, type, parts, size } = await writeBundle(rest[0], rest[1], { libraries })
console.log(`Bundled ${type} in ${output}: ${parts.length} parts from ${from}, ${sized(size)}. Unzip it beside a source and build with --libraries parts.`)
}
} else {
throw new Error(`unknown command "${command}"\n\n${USAGE}`)
}
Expand Down
17 changes: 8 additions & 9 deletions packages/author/lib/author.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,23 @@
* package pins, checked against its index. A tool beside the build takes the libraries as the build
* loads them (`openPack`, `loadLibraries`, and `packageLibraries` for a package's own), its rules
* for text (`markupText`, `plainText`, `allowedTags`), and the check it runs last on what it writes,
* for saved content (`checkContent`). This is the entry for Node: `./core` is what of it needs no
* Node, and `./browser` builds in a page.
* for saved content (`checkContent`). A content type's bundle (`writeBundle`) carries the parts a
* build of it reads, for a build where the pack cannot be installed. This is the entry for Node:
* `./core` is what of it needs no Node, and `./browser` builds in a page.
*/

import { contentTypes } from './bundle.mjs'
import { openPack } from './node.mjs'
import { schemaLines, schemaMarkdown } from './schema.mjs'
import { LAYOUTS } from './types.mjs'

export * from './core.mjs'
export { buildPackage, openPack } from './node.mjs'
export { buildPackage, openPack, writeBundle, writeBundles } from './node.mjs'

/**
* @typedef {import('./node.mjs').BuildResult} BuildResult
* @typedef {import('./node.mjs').BuildOptions} BuildOptions
* @typedef {import('./node.mjs').BundleResult} BundleResult
*/

/**
Expand All @@ -34,12 +37,8 @@ export async function schemaOf(machineName, options = {}) {
try {
const entry = pack.newest(machineName)
if (!entry) {
const runnable = []
for (const each of pack.libraries) {
const part = await pack.open(each)
if (part.library.runnable) runnable.push(`${each.machineName} (${part.library.title})`)
}
throw new Error(`The library pack has no ${machineName}. Its content types: ${[...new Set(runnable)].sort().join(', ')}`)
const types = (await contentTypes(pack)).map((part) => `${part.library.machineName} (${part.library.title})`)
throw new Error(`The library pack has no ${machineName}. Its content types: ${types.join(', ')}`)
}
const part = await pack.open(entry)
if (!part.semantics) throw new Error(`${machineName} has no semantics: it is not a content type an author fills in`)
Expand Down
116 changes: 116 additions & 0 deletions packages/author/lib/bundle.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { librariesNamedIn, planPack } from '@missing-elements/h5p-offline-player/plan'
import { libraryOptions } from './generator.mjs'
import { installClosure } from './install.mjs'

/**
* A content type's bundle: the parts of the pack a build of that type can read, whatever its source
* holds, for a build where the pack cannot be installed and one file per type can be fetched, as in
* ChatGPT's sandbox (#166). It is the build's own rules applied to the most a source of the type can
* name: the content type and every library its library fields offer or give as their default, those
* libraries' in turn; what they play with, as the player plans it (`planPack`); and what an H5P
* site's upload check wants with them (`installClosure`). Both rules only ever add for more roots,
* so whatever one source names is within the bundle.
*
* An addon is not in it. A build adds one where a source's text matches its pattern, H5P.MathDisplay
* (2.8 MB) where the text holds math, which would more than double most bundles. A bundle carries
* the pack's whole index, so a build from it plans as it would from the pack, and either writes the
* same package or stops at the part it lacks, by name.
*/

/**
* @typedef {import('./pack.mjs').Pack} Pack
* @typedef {import('./pack.mjs').IndexEntry} IndexEntry
* @typedef {import('./pack.mjs').PartLibrary} PartLibrary
*/

/**
* The pack's content types: each runnable library at the newest version the pack serves, the one a
* source's `type` builds.
* @param {Pack} pack
* @returns {Promise<PartLibrary[]>} by machine name
*/
export async function contentTypes(pack) {
/** @type {Map<string, PartLibrary>} */
const found = new Map()
for (const { machineName } of pack.libraries) {
if (found.has(machineName)) continue
const entry = pack.newest(machineName)
if (!entry || entry.installOnly) continue
const part = await pack.open(entry)
if (part.library.runnable) found.set(machineName, part)
}
return [...found.values()].sort((a, b) => a.folder.localeCompare(b.folder))
}

/**
* Every library field of semantics, in groups and lists at any depth.
* @param {unknown} semantics
* @returns {any[]}
*/
function libraryFields(semantics) {
/** @type {any[]} */
const found = []
/** @param {any} field */
const walk = (field) => {
if (!field || typeof field !== 'object') return
if (field.type === 'library') found.push(field)
if (Array.isArray(field.fields)) field.fields.forEach(walk)
walk(field.field)
}
if (Array.isArray(semantics)) semantics.forEach(walk)
return found
}

/**
* The parts of a content type's bundle: the content type's first, then what it plays with, then
* what an H5P site's upload check wants, each once.
* @param {Pack} pack
* @param {string} type a content type's machine name, such as `H5P.QuestionSet`, as a source names it
* @returns {Promise<IndexEntry[]>}
*/
export async function bundleLibraries(pack, type) {
const main = pack.newest(String(type).trim())
if (!main || main.installOnly) {
const known = (await contentTypes(pack)).map((part) => part.library.machineName)
throw new Error(`The library pack has no content type ${type}. Its content types: ${known.join(', ')}`)
}
if (!(await pack.open(main)).library.runnable) {
throw new Error(`${main.machineName} is not a content type on its own but a library content types use: bundle the content type you build`)
}
// What content of the type can hold: the libraries its library fields offer, and theirs in turn.
/** @type {Map<string, IndexEntry>} */
const content = new Map()
const queue = [main]
while (queue.length) {
const entry = /** @type {IndexEntry} */ (queue.shift())
if (content.has(entry.folder)) continue
content.set(entry.folder, entry)
for (const field of libraryFields((await pack.open(entry)).semantics)) {
const offered = [
...libraryOptions(field),
...librariesNamedIn(field.default).map(({ machineName, majorVersion }) => ({ machineName, major: Number(majorVersion) }))
]
// Every major the pack has of the name, not only the one offered: a source names a library
// without its version, and `loadLibraries` then opens each major the pack has of it.
for (const { machineName } of offered) {
for (const major of new Set(pack.libraries.filter((entry) => entry.machineName === machineName).map((entry) => entry.majorVersion))) {
const found = pack.newest(machineName, major)
if (found && !found.installOnly) queue.push(found)
}
}
}
}
const roots = [...content.values()].map(({ machineName, majorVersion, minorVersion }) => ({ machineName, majorVersion, minorVersion }))
// As the build plans a package (`closure` in build.mjs), with no parameters: an addon is added
// only for text that matches its pattern, and these are left out.
const planned = planPack({ own: [], provided: [], roots, params: undefined, pack: pack.libraries, refresh: false, fill: true })?.parts ?? []
/** @type {Map<string, PartLibrary>} */
const selected = new Map()
for (const { folder } of planned) {
const entry = pack.libraries.find((each) => each.folder === folder)
if (entry) selected.set(folder, await pack.open(entry))
}
const install = await installClosure(pack, selected)
const parts = [main, ...[...selected.values(), ...install.values()].map((part) => part.index)]
return [...new Map(parts.map((entry) => [entry.folder, entry])).values()]
}
7 changes: 4 additions & 3 deletions packages/author/lib/core.mjs
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
/**
* @missing-elements/h5p-author/core: what the package does that needs neither Node nor a page, for
* a tool that runs beside a build in either: the generator, the libraries loaded as a build loads
* them from any source of them, the check of saved content, and the rules for text. The package's
* entry for Node (`.`) and the one for a page (`./browser`) carry all of it, each with its own
* `buildPackage` and `openPack`.
* them from any source of them, the parts a content type's build can read, the check of saved
* content, and the rules for text. The package's entry for Node (`.`) and the one for a page
* (`./browser`) carry all of it, each with its own `buildPackage` and `openPack`.
*/

export { bundleLibraries, contentTypes } from './bundle.mjs'
export { checkContent } from './check.mjs'
export { ContentGenerator, librariesIn, libraryString } from './generator.mjs'
export { missingForInstall } from './install.mjs'
Expand Down
2 changes: 1 addition & 1 deletion packages/author/lib/generator.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -753,7 +753,7 @@ export function libraryString(/** @type {any} */ library) {
}

/** @param {any} field @returns {{ machineName: string, major: number, minor: number }[]} */
function libraryOptions(field) {
export function libraryOptions(field) {
return (field?.options ?? [])
.map((/** @type {any} */ option) => /^(\S+) (\d+)\.(\d+)$/.exec(typeof option === 'string' ? option : option?.name ?? ''))
.filter(Boolean)
Expand Down
Loading
Loading