diff --git a/AGENTS.md b/AGENTS.md index b968e28f..160b9bc2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1053,8 +1053,12 @@ element, and the element acts. That relay is why `frame-document.ts` has a `mess wherever the source is the pack, `refresh-libraries="off"` to turn it off: a library the package carries plays at the pack's version when the pack has the same major at a higher minor, or the same minor at a higher patch; equal, older or only another major keeps the package's own, with - nothing downloaded. The rule needs each own `library.json` for the patch, so on a host with - `Range` it waits for the deep index after the warm, where they are cached (before it, names give + nothing downloaded. So does a library that a bundle listed before the pack already has at that + major: the bundle would answer for the folder withheld, under the part's own folder name, which + then is never planned, or at a lower minor, which a request resolves to first. Until #58 it did: + the bundle's version played, and the pack's was reported. The rule needs each own + `library.json` for the patch, so on a host with `Range` it waits for the deep index after the + warm, where they are cached (before it, names give major and minor and nothing is refreshed). A refreshed folder is withheld from the package's own index whole (`superseded`, as `untrusted` withholds), so requests for it resolve to the part's folder by the compatible-version rule `get()` already applies — exact for a patch, the newer @@ -1076,7 +1080,11 @@ element, and the element acts. That relay is why `frame-document.ts` has a `mess worker against the index's SHA-256 once downloaded (`checkHash`, read back from the store whole, bounded by the largest part, a few MB) and refused, its cache dropped, on a mismatch; a part's id is its hash (`partPkgId`), so a part kept before is used with no request, by any package, - online or not, and needs no probe. The pinned index is in the worker script, so jsDelivr cannot + online or not, and needs no probe. A match is recorded on the part (`ChunkMeta.checked`), and + complete without it is not enough: a Jobs worker stopped between the last write and the check + left a part whole and unchecked, which every later load took as checked (#58). A part without + the mark — that, or one a player from before kept — is checked on its next use and downloaded + again if it fails. The pinned index is in the worker script, so jsDelivr cannot change the code a host runs; a host's own copy is the host's choice. **Failure**: a refresh that cannot be had — a part that will not download, or does not match — is given up and the package plays as exported, never failing a load that would have played; a part needed for a gap that @@ -2133,8 +2141,9 @@ SPA-fallback trap described above cannot happen there. because the European hosts a data protection officer accepts read none of the first two and the docs had named only US providers. nginx's `add_header` in a `location` replaces the inherited headers, so each location in the file repeats the policy. `--packages` - turns the default `connect-src 'self' https:` into exactly the hosts given, and `config.js` - hands the same list to the page, which refuses an `src` or a library URL off it by name + turns the default `connect-src 'self' https:` into exactly the hosts given — each a plain host, + since the URL parser takes `*` and `"` in one and the page compares exact origins (#58) — and + `config.js` hands the same list to the page, which refuses an `src` or a library URL off it by name before fetching anything. (Until 2026-10-10 it also refused `hub` unless `https://api.h5p.org` was listed; a site without its own pack needs `https://cdn.jsdelivr.net` listed for `pack`, and `hub`, which meant `pack` from then to embed 0.8, is no longer a token.) The page checks only the address it is diff --git a/packages/embed/README.md b/packages/embed/README.md index 10911f3b..66b8deae 100644 --- a/packages/embed/README.md +++ b/packages/embed/README.md @@ -30,7 +30,7 @@ a subdomain shares your site's cookies. | Option | Effect | |---|---| -| `--packages ` | Play packages and fetch library bundles only from these origins (the site's own is always allowed). Add `https://cdn.jsdelivr.net` for `libraries=pack` on a site built with `--no-libraries`. Default: any https host | +| `--packages ` | Play packages and fetch library bundles only from these origins (the site's own is always allowed). Each host on its own, with no wildcard: the page compares exact origins. Add `https://cdn.jsdelivr.net` for `libraries=pack` on a site built with `--no-libraries`. Default: any https host | | `--ancestors ` | Only these sites may frame the page (`frame-ancestors`, header only). Default: any site | | `--default-libraries ` | The `libraries` value for addresses that name none: `pack` or URLs. Checked against `--packages`. Default: none, so exports without libraries are refused unless the address asks | | `--no-libraries` | Leave out the library pack, 13.8 MB in `libraries/`, one archive per library, of which a learner's browser fetches only those a package needs; `libraries=pack` then fetches them from jsDelivr, where allowed | diff --git a/packages/embed/lib/build.mjs b/packages/embed/lib/build.mjs index 04988033..3734011b 100644 --- a/packages/embed/lib/build.mjs +++ b/packages/embed/lib/build.mjs @@ -35,7 +35,10 @@ function installed(specifier, hint) { /** * An origin as the CSP and the page compare it: `https://host[:port]`, or plain http on a loopback * address for trying it locally. Anything with a path, a query or a trailing slash is refused - * rather than trimmed, so a typo is not silently a different policy. + * rather than trimmed, so a typo is not silently a different policy. The host is held to what a + * CSP host is, letters, digits and hyphens between dots: the URL parser takes `*`, `;` and `"` in + * a host, and a wildcard would allow what the page then refuses, by exact origin, on every load, + * while a quote would end the policy in the files that quote it (#58). */ export function parseOrigin(value) { let url @@ -48,6 +51,9 @@ export function parseOrigin(value) { if (url.protocol !== 'https:' && !(url.protocol === 'http:' && loopback)) { throw new EmbedError(`Not an https origin: ${value}`) } + if (!loopback && !/^[a-z0-9-]+(?:\.[a-z0-9-]+)*$/.test(url.hostname)) { + throw new EmbedError(`Not an origin: ${value}. A host is letters, digits, hyphens and dots, and each host is listed on its own, with no wildcard`) + } if (url.origin !== value) throw new EmbedError(`Not an origin: ${value}. Write it as ${url.origin}, with no path or trailing slash`) return url.origin } diff --git a/packages/embed/tests/build.test.ts b/packages/embed/tests/build.test.ts index f310e29f..27ebe9c0 100644 --- a/packages/embed/tests/build.test.ts +++ b/packages/embed/tests/build.test.ts @@ -34,6 +34,15 @@ describe('origins', () => { expect(() => parseOrigin('cdn.example.org')).toThrow(EmbedError) }) + it('refuses a host the URL parser takes and a policy cannot: a wildcard, a quote, a semicolon', () => { + // A wildcard would allow what the page refuses by exact origin; a quote would end the policy + // in nginx.conf, the Caddyfile, .htaccess and the (#58). + expect(() => parseOrigin('https://*.cdn.example.org')).toThrow(/no wildcard/) + expect(() => parseOrigin('https://cdn.example.org"x')).toThrow(EmbedError) + expect(() => parseOrigin('https://cdn.example.org;x')).toThrow(EmbedError) + expect(parseOrigin('https://192.0.2.10')).toBe('https://192.0.2.10') + }) + it('reads a list given once, separated, or several times', () => { expect(parseOrigins(null)).toBeNull() expect(parseOrigins(['https://a.example,https://b.example', 'https://c.example https://a.example'])).toEqual([ diff --git a/packages/player/src/jobs/jobs-worker.ts b/packages/player/src/jobs/jobs-worker.ts index 275418f4..09bc126c 100644 --- a/packages/player/src/jobs/jobs-worker.ts +++ b/packages/player/src/jobs/jobs-worker.ts @@ -1,6 +1,6 @@ /// import { ARCHIVE_ENTRY, CHUNK_SIZE, INPUT_LIVENESS_MS, WARM_ENTRY } from '../shared/constants' -import { ChunkStore, QuotaError, announceQueued, announceRunning, isQuotaError } from '../shared/chunk-store' +import { ChunkStore, QuotaError, announceQueued, announceRunning, isQuotaError, type ChunkMeta } from '../shared/chunk-store' import { installEvictionPolicy } from '../shared/eviction' import { LocalHeaderScanner } from '../shared/forward-index' import { DEFLATE, STORED, readDataStart } from '../shared/local-header' @@ -216,6 +216,17 @@ async function downloadArchive( const existing = await store.getArchiveMeta() if (existing?.complete) { + // A part kept whole but never checked — its worker stopped between the last write and the + // check, or a player from before parts were marked kept it — is checked now, and downloaded + // again if it fails. + if (sha256 && existing.checked !== sha256) { + try { + await checkHash(store, existing, sha256, source.url) + } catch { + await store.destroy() + return downloadArchive(pkgId, source, signal, afresh, sha256) + } + } send({ type: 'done', pkgId }) return } @@ -329,7 +340,7 @@ async function downloadArchive( return downloadArchive(pkgId, source, signal, true, sha256) } if (!meta?.complete) throw new PlayerError('bad-archive', 'The downloaded archive could not be kept in the browser storage') - if (sha256) await checkHash(store, meta.size ?? meta.available, sha256, source.url) + if (sha256) await checkHash(store, meta, sha256, source.url) send({ type: 'done', pkgId }) } @@ -338,11 +349,17 @@ async function downloadArchive( * the pack's index says, dropping what was stored: the part is library code the host's site will * run, and the index — the one this release pins, or the host's — is what names it. Read whole, * which is the one buffering here and is bounded by the pack's largest part, a few megabytes; - * nothing streams SHA-256 in a browser. Once kept, a part is known by its hash and not checked again. + * nothing streams SHA-256 in a browser. A match is recorded on the part, which is then known by + * its hash and not checked again. */ -async function checkHash(store: ChunkStore, size: number, expected: string, url: string): Promise { +async function checkHash(store: ChunkStore, meta: ChunkMeta, expected: string, url: string): Promise { + const size = meta.size ?? meta.available const bytes = await new Response(store.readRange(ARCHIVE_ENTRY, { start: 0, end: size - 1 })).arrayBuffer() - if (hex(new Uint8Array(await crypto.subtle.digest('SHA-256', bytes))) === expected) return + if (hex(new Uint8Array(await crypto.subtle.digest('SHA-256', bytes))) === expected) { + // Best-effort: a store too full to take the mark leaves the part to be checked on its next use. + await store.setMeta(ARCHIVE_ENTRY, { ...meta, checked: expected }).catch(() => {}) + return + } await store.destroy() throw new PlayerError('bad-archive', `${url} is not the library the pack's index names: its bytes do not match the index's hash`) } diff --git a/packages/player/src/shared/chunk-store.ts b/packages/player/src/shared/chunk-store.ts index 47fba921..735eb530 100644 --- a/packages/player/src/shared/chunk-store.ts +++ b/packages/player/src/shared/chunk-store.ts @@ -28,6 +28,12 @@ export interface ChunkMeta { * the entry up again once the failure is old enough to be worth retesting. */ error?: EntryFailure + /** + * For a part of the library pack, the SHA-256 its bytes were found to match. Complete is not + * enough: a Jobs worker stopped between the last write and the check leaves a part whole and + * unchecked, and it is library code (#58). + */ + checked?: string } export interface EntryFailure { diff --git a/packages/player/src/sw/pack-plan.ts b/packages/player/src/sw/pack-plan.ts index 8b33f268..fb189fcb 100644 --- a/packages/player/src/sw/pack-plan.ts +++ b/packages/player/src/sw/pack-plan.ts @@ -96,6 +96,11 @@ export function planPack(input: PlanInput): PackPlan | null { if (library.patch === null || library.major === null) continue const candidate = byLibrary.get(libraryKey(library.machineName, library.major)) if (!candidate || !newer(candidate, library)) continue + // A bundle listed before the pack has this major already, and would answer for the folder + // withheld: under the part's own folder name, which then is never planned, or at a lower + // minor, the first a request resolves to. The package keeps its own, as a bundle there + // fills gaps only, rather than playing the bundle's version reported as the pack's (#58). + if (input.provided.some((each) => each.machineName === library.machineName && each.major === library.major)) continue supersede(library, candidate) queue.push(...plan(candidate)) } diff --git a/packages/player/tests/browser/pack.test.ts b/packages/player/tests/browser/pack.test.ts index 7d5da3ab..6a33bbad 100644 --- a/packages/player/tests/browser/pack.test.ts +++ b/packages/player/tests/browser/pack.test.ts @@ -1,8 +1,11 @@ import { describe, expect, it } from 'vitest' import '../../src/h5p-offline-player' import type { H5PPlayerElement } from '../../src/h5p-offline-player' +import { ChunkStore } from '../../src/shared/chunk-store' +import { ARCHIVE_ENTRY } from '../../src/shared/constants' import * as db from '../../src/shared/idb' import { packageLockName } from '../../src/shared/locks' +import { partPkgId } from '../../src/shared/pkg-id' import { FIXTURES, clearPackageCaches, createPlayer, frameWindow, messageOf, play, settle, waitForSettled } from './utils' /** @@ -128,6 +131,26 @@ describe('the library pack, split into parts', () => { expect(player.libraryBundle).toBeNull() }) + it('checks a part kept whole but never checked, and downloads it again when it does not match', async () => { + // What a Jobs worker stopped between a part's last write and its check leaves behind: the + // part complete, under its hash's id, holding other bytes — here another part's. Taken as + // checked for being whole, it was attached as the library (#58). + const index = (await (await fetch(`${PACK}index.json`)).json()) as { libraries: Array<{ folder: string; sha256: string }> } + const part = index.libraries.find(({ folder }) => folder === 'H5P.OfflineTest-1.2')! + const other = index.libraries.find(({ folder }) => folder !== part.folder)! + const wrong = new Uint8Array(await (await fetch(`${PACK}${other.folder}.h5p`)).arrayBuffer()) + const id = await partPkgId(part.sha256) + const store = new ChunkStore(id) + await store.destroy() + await store.putChunk(ARCHIVE_ENTRY, 0, wrong) + await store.setMeta(ARCHIVE_ENTRY, { size: wrong.length, available: wrong.length, complete: true }) + + const player = await play(fresh(FIXTURES.needsLibraries, 'unchecked'), withPack()) + expect(messageOf(player)).toBe('Libraries came from somewhere else. [1.1] [1.2]') + // A store of its own: the worker deleted the cache this one holds, and wrote a new one. + expect((await new ChunkStore(id).getArchiveMeta())?.checked).toBe(part.sha256) + }) + it("keeps what the package carries when the host's copy of the pack cannot be read", async () => { const missing = withPack({ 'pack-base': '/no-range/no-such-pack/' }) diff --git a/packages/player/tests/unit/pack-plan.test.ts b/packages/player/tests/unit/pack-plan.test.ts index 52d6f195..34b746a6 100644 --- a/packages/player/tests/unit/pack-plan.test.ts +++ b/packages/player/tests/unit/pack-plan.test.ts @@ -106,6 +106,20 @@ describe('refreshing what a package carries', () => { expect(folders(result)).toEqual(['H5P.Question-1.5']) expect(result?.superseded).toEqual(['H5P.Question-1.4']) }) + + it('keeps the package\'s own where a bundle listed before the pack has the library already', () => { + // `libraries="/bundle.h5p pack"`: the bundle was attached for a gap and is consulted before + // the pack's parts, so it would answer for the folder a refresh withheld (#58). + const sameFolder = [{ folder: 'H5P.Video-1.6', machineName: 'H5P.Video', major: 1, minor: 6 }] + expect(plan({ own: [carried('H5P.Video', 1, 6, 66)], provided: sameFolder, roots: dependencies([['H5P.Video', 1, 6]]) })).toBeNull() + + // At a minor between the package's and the pack's, the first a request for the package's + // resolves to: the part would download and never answer. + const between = [{ folder: 'H5P.Question-1.5', machineName: 'H5P.Question', major: 1, minor: 5 }] + const own = [carried('H5P.Question', 1, 4, 1)] + const pack = [...PACK.filter(({ machineName }) => machineName !== 'H5P.Question'), inPack('H5P.Question', 1, 6, 0)] + expect(plan({ own, provided: between, roots: dependencies([['H5P.Question', 1, 4]]), pack })).toBeNull() + }) }) describe('supplying what a package lacks', () => { diff --git a/packages/verify/lib/verify.mjs b/packages/verify/lib/verify.mjs index db5fd1df..85b8f857 100644 --- a/packages/verify/lib/verify.mjs +++ b/packages/verify/lib/verify.mjs @@ -338,12 +338,13 @@ export function contentChanges(written, played) { } for (const key of Object.keys(a)) { const next = Array.isArray(a) ? `${path}[${key}]` : path ? `${path}.${key}` : key - if (!(key in b)) { + // Own keys only: `in` finds `constructor` and `toString` on every object (#58). + if (!Object.hasOwn(b, key)) { if (!isEmpty(a[key])) changes.push({ path: next, change: 'removed' }) } else walk(a[key], b[key], next) } for (const key of Object.keys(b)) { - if (!(key in a)) changes.push({ path: Array.isArray(b) ? `${path}[${key}]` : path ? `${path}.${key}` : key, change: 'added' }) + if (!Object.hasOwn(a, key)) changes.push({ path: Array.isArray(b) ? `${path}[${key}]` : path ? `${path}.${key}` : key, change: 'added' }) } return } diff --git a/packages/verify/tests/verify.test.ts b/packages/verify/tests/verify.test.ts index 1a50421a..70b1998a 100644 --- a/packages/verify/tests/verify.test.ts +++ b/packages/verify/tests/verify.test.ts @@ -137,6 +137,16 @@ describe('contentChanges', () => { { path: 'html', change: 'text rewritten' } ]) }) + + it('reports a removed or added field named as a member of every object as such', () => { + // `in` found `constructor` on the object it was removed from, and compared the text with + // Object itself (#58). + expect(contentChanges({ constructor: 'x', toString: 'y' }, {})).toEqual([ + { path: 'constructor', change: 'removed' }, + { path: 'toString', change: 'removed' } + ]) + expect(contentChanges({}, { valueOf: 1 })).toEqual([{ path: 'valueOf', change: 'added' }]) + }) }) describe('h5p-verify', () => {