Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1053,8 +1053,12 @@ element, and the element acts. That relay is why `frame-document.ts` has a `mess
wherever the source is the pack, `refresh-libraries="off"` to turn it off: a library the package
carries plays at the pack's version when the pack has the same major at a higher minor, or the
same minor at a higher patch; equal, older or only another major keeps the package's own, with
nothing downloaded. The rule needs each own `library.json` for the patch, so on a host with
`Range` it waits for the deep index after the warm, where they are cached (before it, names give
nothing downloaded. So does a library that a bundle listed before the pack already has at that
major: the bundle would answer for the folder withheld, under the part's own folder name, which
then is never planned, or at a lower minor, which a request resolves to first. Until #58 it did:
the bundle's version played, and the pack's was reported. The rule needs each own
`library.json` for the patch, so on a host with `Range` it waits for the deep index after the
warm, where they are cached (before it, names give
major and minor and nothing is refreshed). A refreshed folder is withheld from the package's own
index whole (`superseded`, as `untrusted` withholds), so requests for it resolve to the part's
folder by the compatible-version rule `get()` already applies — exact for a patch, the newer
Expand All @@ -1076,7 +1080,11 @@ element, and the element acts. That relay is why `frame-document.ts` has a `mess
worker against the index's SHA-256 once downloaded (`checkHash`, read back from the store whole,
bounded by the largest part, a few MB) and refused, its cache dropped, on a mismatch; a part's
id is its hash (`partPkgId`), so a part kept before is used with no request, by any package,
online or not, and needs no probe. The pinned index is in the worker script, so jsDelivr cannot
online or not, and needs no probe. A match is recorded on the part (`ChunkMeta.checked`), and
complete without it is not enough: a Jobs worker stopped between the last write and the check
left a part whole and unchecked, which every later load took as checked (#58). A part without
the mark — that, or one a player from before kept — is checked on its next use and downloaded
again if it fails. The pinned index is in the worker script, so jsDelivr cannot
change the code a host runs; a host's own copy is the host's choice. **Failure**: a refresh that
cannot be had — a part that will not download, or does not match — is given up and the package
plays as exported, never failing a load that would have played; a part needed for a gap that
Expand Down Expand Up @@ -2133,8 +2141,9 @@ SPA-fallback trap described above cannot happen there.
because the European hosts a data protection officer accepts read none of the first two and
the docs had named only US providers. nginx's `add_header` in a `location` replaces the
inherited headers, so each location in the file repeats the policy. `--packages`
turns the default `connect-src 'self' https:` into exactly the hosts given, and `config.js`
hands the same list to the page, which refuses an `src` or a library URL off it by name
turns the default `connect-src 'self' https:` into exactly the hosts given — each a plain host,
since the URL parser takes `*` and `"` in one and the page compares exact origins (#58) — and
`config.js` hands the same list to the page, which refuses an `src` or a library URL off it by name
before fetching anything. (Until 2026-10-10 it also refused `hub` unless `https://api.h5p.org`
was listed; a site without its own pack needs `https://cdn.jsdelivr.net` listed for `pack`, and
`hub`, which meant `pack` from then to embed 0.8, is no longer a token.) The page checks only the address it is
Expand Down
2 changes: 1 addition & 1 deletion packages/embed/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ a subdomain shares your site's cookies.

| Option | Effect |
|---|---|
| `--packages <origins>` | Play packages and fetch library bundles only from these origins (the site's own is always allowed). Add `https://cdn.jsdelivr.net` for `libraries=pack` on a site built with `--no-libraries`. Default: any https host |
| `--packages <origins>` | Play packages and fetch library bundles only from these origins (the site's own is always allowed). Each host on its own, with no wildcard: the page compares exact origins. Add `https://cdn.jsdelivr.net` for `libraries=pack` on a site built with `--no-libraries`. Default: any https host |
| `--ancestors <origins>` | Only these sites may frame the page (`frame-ancestors`, header only). Default: any site |
| `--default-libraries <sources>` | The `libraries` value for addresses that name none: `pack` or URLs. Checked against `--packages`. Default: none, so exports without libraries are refused unless the address asks |
| `--no-libraries` | Leave out the library pack, 13.8 MB in `libraries/`, one archive per library, of which a learner's browser fetches only those a package needs; `libraries=pack` then fetches them from jsDelivr, where allowed |
Expand Down
8 changes: 7 additions & 1 deletion packages/embed/lib/build.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,10 @@ function installed(specifier, hint) {
/**
* An origin as the CSP and the page compare it: `https://host[:port]`, or plain http on a loopback
* address for trying it locally. Anything with a path, a query or a trailing slash is refused
* rather than trimmed, so a typo is not silently a different policy.
* rather than trimmed, so a typo is not silently a different policy. The host is held to what a
* CSP host is, letters, digits and hyphens between dots: the URL parser takes `*`, `;` and `"` in
* a host, and a wildcard would allow what the page then refuses, by exact origin, on every load,
* while a quote would end the policy in the files that quote it (#58).
*/
export function parseOrigin(value) {
let url
Expand All @@ -48,6 +51,9 @@ export function parseOrigin(value) {
if (url.protocol !== 'https:' && !(url.protocol === 'http:' && loopback)) {
throw new EmbedError(`Not an https origin: ${value}`)
}
if (!loopback && !/^[a-z0-9-]+(?:\.[a-z0-9-]+)*$/.test(url.hostname)) {
throw new EmbedError(`Not an origin: ${value}. A host is letters, digits, hyphens and dots, and each host is listed on its own, with no wildcard`)
}
if (url.origin !== value) throw new EmbedError(`Not an origin: ${value}. Write it as ${url.origin}, with no path or trailing slash`)
return url.origin
}
Expand Down
9 changes: 9 additions & 0 deletions packages/embed/tests/build.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,15 @@ describe('origins', () => {
expect(() => parseOrigin('cdn.example.org')).toThrow(EmbedError)
})

it('refuses a host the URL parser takes and a policy cannot: a wildcard, a quote, a semicolon', () => {
// A wildcard would allow what the page refuses by exact origin; a quote would end the policy
// in nginx.conf, the Caddyfile, .htaccess and the <meta> (#58).
expect(() => parseOrigin('https://*.cdn.example.org')).toThrow(/no wildcard/)
expect(() => parseOrigin('https://cdn.example.org"x')).toThrow(EmbedError)
expect(() => parseOrigin('https://cdn.example.org;x')).toThrow(EmbedError)
expect(parseOrigin('https://192.0.2.10')).toBe('https://192.0.2.10')
})

it('reads a list given once, separated, or several times', () => {
expect(parseOrigins(null)).toBeNull()
expect(parseOrigins(['https://a.example,https://b.example', 'https://c.example https://a.example'])).toEqual([
Expand Down
27 changes: 22 additions & 5 deletions packages/player/src/jobs/jobs-worker.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/// <reference lib="webworker" />
import { ARCHIVE_ENTRY, CHUNK_SIZE, INPUT_LIVENESS_MS, WARM_ENTRY } from '../shared/constants'
import { ChunkStore, QuotaError, announceQueued, announceRunning, isQuotaError } from '../shared/chunk-store'
import { ChunkStore, QuotaError, announceQueued, announceRunning, isQuotaError, type ChunkMeta } from '../shared/chunk-store'
import { installEvictionPolicy } from '../shared/eviction'
import { LocalHeaderScanner } from '../shared/forward-index'
import { DEFLATE, STORED, readDataStart } from '../shared/local-header'
Expand Down Expand Up @@ -216,6 +216,17 @@ async function downloadArchive(
const existing = await store.getArchiveMeta()

if (existing?.complete) {
// A part kept whole but never checked — its worker stopped between the last write and the
// check, or a player from before parts were marked kept it — is checked now, and downloaded
// again if it fails.
if (sha256 && existing.checked !== sha256) {
try {
await checkHash(store, existing, sha256, source.url)
} catch {
await store.destroy()
return downloadArchive(pkgId, source, signal, afresh, sha256)
}
}
send({ type: 'done', pkgId })
return
}
Expand Down Expand Up @@ -329,7 +340,7 @@ async function downloadArchive(
return downloadArchive(pkgId, source, signal, true, sha256)
}
if (!meta?.complete) throw new PlayerError('bad-archive', 'The downloaded archive could not be kept in the browser storage')
if (sha256) await checkHash(store, meta.size ?? meta.available, sha256, source.url)
if (sha256) await checkHash(store, meta, sha256, source.url)
send({ type: 'done', pkgId })
}

Expand All @@ -338,11 +349,17 @@ async function downloadArchive(
* the pack's index says, dropping what was stored: the part is library code the host's site will
* run, and the index — the one this release pins, or the host's — is what names it. Read whole,
* which is the one buffering here and is bounded by the pack's largest part, a few megabytes;
* nothing streams SHA-256 in a browser. Once kept, a part is known by its hash and not checked again.
* nothing streams SHA-256 in a browser. A match is recorded on the part, which is then known by
* its hash and not checked again.
*/
async function checkHash(store: ChunkStore, size: number, expected: string, url: string): Promise<void> {
async function checkHash(store: ChunkStore, meta: ChunkMeta, expected: string, url: string): Promise<void> {
const size = meta.size ?? meta.available
const bytes = await new Response(store.readRange(ARCHIVE_ENTRY, { start: 0, end: size - 1 })).arrayBuffer()
if (hex(new Uint8Array(await crypto.subtle.digest('SHA-256', bytes))) === expected) return
if (hex(new Uint8Array(await crypto.subtle.digest('SHA-256', bytes))) === expected) {
// Best-effort: a store too full to take the mark leaves the part to be checked on its next use.
await store.setMeta(ARCHIVE_ENTRY, { ...meta, checked: expected }).catch(() => {})
return
}
await store.destroy()
throw new PlayerError('bad-archive', `${url} is not the library the pack's index names: its bytes do not match the index's hash`)
}
Expand Down
6 changes: 6 additions & 0 deletions packages/player/src/shared/chunk-store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@ export interface ChunkMeta {
* the entry up again once the failure is old enough to be worth retesting.
*/
error?: EntryFailure
/**
* For a part of the library pack, the SHA-256 its bytes were found to match. Complete is not
* enough: a Jobs worker stopped between the last write and the check leaves a part whole and
* unchecked, and it is library code (#58).
*/
checked?: string
}

export interface EntryFailure {
Expand Down
5 changes: 5 additions & 0 deletions packages/player/src/sw/pack-plan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,11 @@ export function planPack(input: PlanInput): PackPlan | null {
if (library.patch === null || library.major === null) continue
const candidate = byLibrary.get(libraryKey(library.machineName, library.major))
if (!candidate || !newer(candidate, library)) continue
// A bundle listed before the pack has this major already, and would answer for the folder
// withheld: under the part's own folder name, which then is never planned, or at a lower
// minor, the first a request resolves to. The package keeps its own, as a bundle there
// fills gaps only, rather than playing the bundle's version reported as the pack's (#58).
if (input.provided.some((each) => each.machineName === library.machineName && each.major === library.major)) continue
supersede(library, candidate)
queue.push(...plan(candidate))
}
Expand Down
23 changes: 23 additions & 0 deletions packages/player/tests/browser/pack.test.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
import { describe, expect, it } from 'vitest'
import '../../src/h5p-offline-player'
import type { H5PPlayerElement } from '../../src/h5p-offline-player'
import { ChunkStore } from '../../src/shared/chunk-store'
import { ARCHIVE_ENTRY } from '../../src/shared/constants'
import * as db from '../../src/shared/idb'
import { packageLockName } from '../../src/shared/locks'
import { partPkgId } from '../../src/shared/pkg-id'
import { FIXTURES, clearPackageCaches, createPlayer, frameWindow, messageOf, play, settle, waitForSettled } from './utils'

/**
Expand Down Expand Up @@ -128,6 +131,26 @@ describe('the library pack, split into parts', () => {
expect(player.libraryBundle).toBeNull()
})

it('checks a part kept whole but never checked, and downloads it again when it does not match', async () => {
// What a Jobs worker stopped between a part's last write and its check leaves behind: the
// part complete, under its hash's id, holding other bytes — here another part's. Taken as
// checked for being whole, it was attached as the library (#58).
const index = (await (await fetch(`${PACK}index.json`)).json()) as { libraries: Array<{ folder: string; sha256: string }> }
const part = index.libraries.find(({ folder }) => folder === 'H5P.OfflineTest-1.2')!
const other = index.libraries.find(({ folder }) => folder !== part.folder)!
const wrong = new Uint8Array(await (await fetch(`${PACK}${other.folder}.h5p`)).arrayBuffer())
const id = await partPkgId(part.sha256)
const store = new ChunkStore(id)
await store.destroy()
await store.putChunk(ARCHIVE_ENTRY, 0, wrong)
await store.setMeta(ARCHIVE_ENTRY, { size: wrong.length, available: wrong.length, complete: true })

const player = await play(fresh(FIXTURES.needsLibraries, 'unchecked'), withPack())
expect(messageOf(player)).toBe('Libraries came from somewhere else. [1.1] [1.2]')
// A store of its own: the worker deleted the cache this one holds, and wrote a new one.
expect((await new ChunkStore(id).getArchiveMeta())?.checked).toBe(part.sha256)
})

it("keeps what the package carries when the host's copy of the pack cannot be read", async () => {
const missing = withPack({ 'pack-base': '/no-range/no-such-pack/' })

Expand Down
14 changes: 14 additions & 0 deletions packages/player/tests/unit/pack-plan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,20 @@ describe('refreshing what a package carries', () => {
expect(folders(result)).toEqual(['H5P.Question-1.5'])
expect(result?.superseded).toEqual(['H5P.Question-1.4'])
})

it('keeps the package\'s own where a bundle listed before the pack has the library already', () => {
// `libraries="/bundle.h5p pack"`: the bundle was attached for a gap and is consulted before
// the pack's parts, so it would answer for the folder a refresh withheld (#58).
const sameFolder = [{ folder: 'H5P.Video-1.6', machineName: 'H5P.Video', major: 1, minor: 6 }]
expect(plan({ own: [carried('H5P.Video', 1, 6, 66)], provided: sameFolder, roots: dependencies([['H5P.Video', 1, 6]]) })).toBeNull()

// At a minor between the package's and the pack's, the first a request for the package's
// resolves to: the part would download and never answer.
const between = [{ folder: 'H5P.Question-1.5', machineName: 'H5P.Question', major: 1, minor: 5 }]
const own = [carried('H5P.Question', 1, 4, 1)]
const pack = [...PACK.filter(({ machineName }) => machineName !== 'H5P.Question'), inPack('H5P.Question', 1, 6, 0)]
expect(plan({ own, provided: between, roots: dependencies([['H5P.Question', 1, 4]]), pack })).toBeNull()
})
})

describe('supplying what a package lacks', () => {
Expand Down
5 changes: 3 additions & 2 deletions packages/verify/lib/verify.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -338,12 +338,13 @@ export function contentChanges(written, played) {
}
for (const key of Object.keys(a)) {
const next = Array.isArray(a) ? `${path}[${key}]` : path ? `${path}.${key}` : key
if (!(key in b)) {
// Own keys only: `in` finds `constructor` and `toString` on every object (#58).
if (!Object.hasOwn(b, key)) {
if (!isEmpty(a[key])) changes.push({ path: next, change: 'removed' })
} else walk(a[key], b[key], next)
}
for (const key of Object.keys(b)) {
if (!(key in a)) changes.push({ path: Array.isArray(b) ? `${path}[${key}]` : path ? `${path}.${key}` : key, change: 'added' })
if (!Object.hasOwn(a, key)) changes.push({ path: Array.isArray(b) ? `${path}[${key}]` : path ? `${path}.${key}` : key, change: 'added' })
}
return
}
Expand Down
10 changes: 10 additions & 0 deletions packages/verify/tests/verify.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,16 @@ describe('contentChanges', () => {
{ path: 'html', change: 'text rewritten' }
])
})

it('reports a removed or added field named as a member of every object as such', () => {
// `in` found `constructor` on the object it was removed from, and compared the text with
// Object itself (#58).
expect(contentChanges({ constructor: 'x', toString: 'y' }, {})).toEqual([
{ path: 'constructor', change: 'removed' },
{ path: 'toString', change: 'removed' }
])
expect(contentChanges({}, { valueOf: 1 })).toEqual([{ path: 'valueOf', change: 'added' }])
})
})

describe('h5p-verify', () => {
Expand Down
Loading