Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 32 additions & 8 deletions design/author.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,38 @@ to 3,000. Twelve things the code does not say by itself:
left as the editor leaves it, asking nothing.
- **Text is stored as the platforms store it.** A plain field is escaped as `htmlspecialchars`
does, existing entities kept, and a tag in it is noted, since a learner sees it as written. A
field with `tags` takes markup with the tags the content check keeps for it (its own, `div`,
`span`, `p`, `br`, and what they imply), checked here rather than filtered later: any other tag,
an event handler, a style the field does not allow, a script address or a comment is a problem
by name. Plain text written into such a field is wrapped as the editor wraps typing: in `<p>`
where the field's `enterMode` is `p` or its tags offer headings, `pre`, `p` or `div`, in `<div>`
otherwise. Every plain default is escaped too, English or translated, and a `null` default (Course
Presentation's colours) is left out, which the content check would otherwise turn into a
warning. Every type file's example plays untrusted with no warning, which is the test of this.
field the editor gives its rich-text widget (`widget: "html"`, or a ShowWhen that draws one) takes
markup with the tags the content check keeps for it (its own, `div`, `span`, `p`, `br`, and what
they imply), checked here rather than filtered later: any other tag, an event handler, a style
the field does not allow, a script address or a comment is a problem by name. Plain text written
into such a field is wrapped as that widget wraps typing: in `<p>` where the field's `enterMode`
is `p` or its tags offer headings, `pre`, `p` or `div`, in `<div>` otherwise. Every plain default
is escaped too, English or translated, and a `null` default (Course Presentation's colours) is
left out, which the content check would otherwise turn into a warning. Every type file's example
plays untrusted with no warning, which is the test of this.
- **`tags` say what the check keeps, the widget what an author writes** (2026-11-09). Up to author
0.5.0 any field that listed `tags` took markup, and plain text in it was wrapped. Five of
Question Set 1.21's fields list `tags` but are plain text boxes in the editor (`ns.Text`, which
stores `htmlspecialchars(value)` and wraps nothing, at the pinned commit): the intro title, the
result message, the progress text and the two old greetings. The set draws the first two with
H5P.Components' `textContent`, so a quiz from ChatGPT showed `<div>World War II: Five Historic
Events</div>` on its cover, and the type file had told the model both fields take HTML.
`takesMarkup` is now the rule, in the build, in its messages and in the type files; saved
content is still checked by `tags`, as the content check keeps it. Of the pack's 2,361 fields,
1,252 are of type `text`, whatever widget draws them, and these five were the only ones of those
that differed: the other 1,096 plain boxes and the 96 rich-text fields (Speak the Words Set's
ShowWhen among them) were stored as their widgets store them, and so were the 38 drawn as colour
selectors, the 6 text areas, the one date and the 10 hidden. The editor puts a default in
unescaped, and the server filters it by `tags`, so defaults still go by `tags`. One rule differs
from the editor, the owner's call the same day: in a plain box whose field lists `tags`
(`plainText`'s `filtered`), an apostrophe is stored as typed. The editor's box writes `&#039;`,
and H5P.Components 1.0.101 (upstream's newest) shows that as written, `Germany&#039;s`, on every
H5P site; the content check filters such a field as markup, which keeps an apostrophe, so a site
changes nothing. A `"` stays `&quot;`, since the editor puts a stored value back into its box as
`value="…"` unescaped (`ns.createText`), where a bare one would cut the text short; `&`, `<` and
`>` stay escaped, as the filter would escape them, and are noted, with "and" and “ ” as the way
round. Umlauts, ß and every other letter are stored as typed everywhere: neither the editor nor
the platforms escape them.
- **Some types need what their editor widget does.** Course Presentation, Interactive Video, Game
Map and Branching Scenario are edited on a slide, a video, a map and a tree of their own, which set
fields the form hides (`widget: "none"`). `lib/types.mjs` does each one's work on the generated
Expand Down
2 changes: 1 addition & 1 deletion packages/author/lib/core.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ export { missingForInstall } from './install.mjs'
export { loadLibraries, packageLibraries } from './libraries.mjs'
export { GENERATED_END, GENERATED_START } from './schema.mjs'
export { SourceError } from './source-error.mjs'
export { allowedTags, markupText, plainText } from './text.mjs'
export { allowedTags, editorWidget, markupText, plainText, takesMarkup } from './text.mjs'

/**
* @typedef {import('./build.mjs').Source} Source
Expand Down
13 changes: 8 additions & 5 deletions packages/author/lib/generator.mjs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { checkMetadata } from './metadata.mjs'
import { markupText, plainText } from './text.mjs'
import { allowedTags, markupText, plainText, takesMarkup } from './text.mjs'

/**
* Content for a content type, made from its own `semantics.json` as H5P's editor saves a form:
Expand Down Expand Up @@ -610,9 +610,12 @@ export class ContentGenerator {
this.report(path, `expected text${typeof written === 'number' ? `, such as "${written}"` : ''}`)
return undefined
}
const stored = field.tags === undefined || field.tags === null
? plainText(written, (message) => this.note(path, message))
: markupText(written, field, (message) => this.report(path, message), { saved: this.saved })
// Written text is stored as the editor's widget for the field stores what an author types;
// saved text, as the content check keeps it, which goes by the field's `tags` alone.
const markup = this.saved ? allowedTags(field) !== null : takesMarkup(field)
const stored = markup
? markupText(written, field, (message) => this.report(path, message), { saved: this.saved })
: plainText(written, (message) => this.note(path, message), { filtered: allowedTags(field) !== null })
const length = Array.from(stored).length
if (typeof field.maxLength === 'number' && length > field.maxLength) this.report(path, `is ${length} characters as stored; it takes at most ${field.maxLength}`)
if (field.regexp?.pattern && !(stored === '' && field.optional)) {
Expand Down Expand Up @@ -811,7 +814,7 @@ function selectValues(field) {
/** A field's kind, as a problem names it. @param {any} field */
function describe(field) {
switch (field.type) {
case 'text': return field.tags ? 'markup text' : 'text'
case 'text': return takesMarkup(field) ? 'markup text' : 'text'
case 'select': return `one of ${selectValues(field).map((value) => JSON.stringify(value)).join(', ')}`
case 'library': return `sub-content: ${optionNames(field).join(', ')}`
case 'image': case 'file': return `a file, { "path": "…" }`
Expand Down
5 changes: 2 additions & 3 deletions packages/author/lib/schema.mjs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { allowedTags } from './text.mjs'
import { editorWidget, takesMarkup } from './text.mjs'

/**
* A content type's fields as an author writes them, from its `semantics.json`: names, kinds,
Expand Down Expand Up @@ -65,8 +65,7 @@ function selectValues(field) {
function kind(field, options, drawn = false) {
switch (field.type) {
case 'text': {
const tags = allowedTags(field)
if (!tags) return field.widget === 'textarea' ? 'text, multi-line' : 'text'
if (!takesMarkup(field)) return editorWidget(field) === 'textarea' ? 'text, multi-line' : 'text'
const own = (Array.isArray(field.tags) ? field.tags : []).filter((/** @type {string} */ tag) => !['div', 'span', 'p', 'br'].includes(tag))
return own.length ? `html: ${own.join(' ')}` : 'html'
}
Expand Down
59 changes: 50 additions & 9 deletions packages/author/lib/text.mjs
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
/**
* Authored text as H5P stores it. A field without `tags` is plain text, which the platforms keep
* escaped (PHP's `htmlspecialchars`, existing entities left alone). A field with `tags` holds
* markup, as H5P's editor writes it into that field: the tags the field names, and the four the
* content check always keeps, `div`, `span`, `p` and `br`. Anything else is refused by name, so a
* source never reaches a learner in a form the platforms would have changed.
* Authored text as H5P stores it. A field H5P's editor gives its rich-text widget holds markup, as
* that widget writes it: the tags the field names, and the four the content check always keeps,
* `div`, `span`, `p` and `br`. Anything else is refused by name, so a source never reaches a
* learner in a form the platforms would have changed. Every other field is plain text, which the
* platforms keep escaped (PHP's `htmlspecialchars`, existing entities left alone), whether or not
* its semantics list `tags` (see `takesMarkup`).
*/

/** An `&` that does not begin an entity. */
Expand All @@ -19,8 +20,34 @@ export function escapeText(value) {
}

/**
* The tags a markup field keeps, as the content check derives them from its semantics: the four
* it always keeps, the field's own, and those one of them implies. `null` for a plain text field.
* The widget H5P's editor draws a field with: its `widget`, or for a ShowWhen, the widget the
* ShowWhen draws inside it, else the field's type (h5peditor.js `getWidgetName`, H5PEditor.ShowWhen).
* @param {any} field
* @returns {string}
*/
export function editorWidget(field) {
if (field.widget === 'showWhen') return field.showWhen?.widget ?? field.type
return field.widget ?? field.type
}

/**
* Whether a text field holds markup: the editor gives it its rich-text widget, `html`, directly or
* inside a ShowWhen, and its semantics list `tags`. Every other text field is a box an author types
* plain text into: `ns.Text` and `ns.Textarea` store it with `htmlspecialchars` and wrap nothing,
* even where the semantics list `tags`, which then tell the content check what to keep, not what
* an author can write. Question Set 1.21's intro title, result message and progress text are such
* fields, and it draws the first two with `textContent`, so markup written there is shown as
* written. A rich-text field without `tags` is plain too: the content check escapes it.
* @param {any} field
*/
export function takesMarkup(field) {
return field.tags !== undefined && field.tags !== null && editorWidget(field) === 'html'
}

/**
* The tags a field keeps, as the content check derives them from its semantics: the four it
* always keeps, the field's own, and those one of them implies. `null` for a field that lists none.
* Which tags the check keeps is not whether an author writes markup there: `takesMarkup` says that.
* @param {any} field
* @returns {Set<string> | null}
*/
Expand Down Expand Up @@ -159,10 +186,24 @@ export function markupText(value, field, report, { saved = false } = {}) {
* A plain text field's value as it is stored: escaped, as the platforms store what an author types
* into it. A tag in it is shown to a learner as written, which is right for a quiz on HTML and a
* mistake otherwise, so it is noted.
*
* `filtered`: the field is a plain text box whose semantics list `tags` (see `takesMarkup`), which
* the content check filters as markup rather than escaping whole. That filter keeps an apostrophe
* as it is, so it is stored as typed: escaped, as the editor's box escapes it (`&#039;`), it is
* shown as written where a content type draws the field with `textContent`, as Question Set 1.21
* draws its cover's title and its results heading. A `"` stays escaped all the same: the editor puts
* a stored value back into its box as `value="…"`, unescaped (`ns.createText`), so a bare one would
* cut the text short there. `&`, `<` and `>` stay escaped, as the filter would escape them, and are
* noted, since such a field shows them as entities.
* @param {string} value
* @param {(message: string) => void} note
* @param {{ filtered?: boolean }} [options]
*/
export function plainText(value, note) {
export function plainText(value, note, { filtered = false } = {}) {
if (/<\/?[a-zA-Z][a-zA-Z0-9]*(\s[^<>]*)?\/?>/.test(value)) note('is plain text, so its tags are shown as written, not as markup')
return escapeText(value)
if (!filtered) return escapeText(value)
if (value.search(BARE_AMPERSAND) !== -1 || /["<>]/.test(value)) {
note('stores &, ", < and > as entities, which a content type that draws this field as plain text shows as written, as Question Set does its title and results heading: write "and", and “ ” for quotes')
}
return value.replace(BARE_AMPERSAND, '&amp;').replaceAll('<', '&lt;').replaceAll('>', '&gt;').replaceAll('"', '&quot;')
}
4 changes: 3 additions & 1 deletion packages/author/lib/translate.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,9 @@ export function translateDefaults(fields, translation) {

/**
* Every plain text default stored as the platforms store it, escaped: a default is what an author
* would have left in the field, and the server escapes it on saving like anything typed there.
* would have left in the field, and the server escapes it on saving like anything typed there. The
* editor puts a default in as it stands, not escaped as its text box escapes typing, so a default in
* a field that lists `tags` is left to the server's filter, which keeps it, in a plain text box too.
* @param {any[]} fields
*/
function escapeDefaults(fields) {
Expand Down
36 changes: 36 additions & 0 deletions packages/author/tests/build.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,42 @@ describe('any content type in the pack', () => {
expect(notes).toEqual([])
})

it('stores a plain text box as plain text, though its field lists tags', async () => {
// Question Set 1.21's intro title and result message list tags, but the editor gives both a
// plain text box, and the set draws both with textContent: a <div> around them was shown.
const { content, notes } = await build({
...META,
type: 'H5P.QuestionSet',
params: {
introPage: { showIntroPage: true, title: 'World War II: Five Historic Events', introduction: 'Five questions on the war.' },
endGame: { message: "Tom's result" },
questions: [{ library: 'H5P.TrueFalse', params: { question: 'The war in Europe ended in 1945.', correct: 'true' } }]
}
})
expect(content.introPage.title).toBe('World War II: Five Historic Events')
// An apostrophe as typed: the content check keeps it there, and the set shows an entity as written.
expect(content.endGame.message).toBe("Tom's result")
// The introduction is the set's rich-text field, wrapped as that widget wraps typing.
expect(content.introPage.introduction).toBe('<p>Five questions on the war.</p>')
expect(notes).toEqual([])

// German as typed, umlauts and ß included; &, " and < stored as entities, and noted.
const german = await build({ ...META, language: 'de', type: 'H5P.QuestionSet', params: {
introPage: { showIntroPage: true, title: 'Größte Ereignisse: "Krieg" & Frieden' },
questions: [{ library: 'H5P.TrueFalse', params: { question: 'Ja?', correct: 'true' } }]
} })
expect(german.content.introPage.title).toBe('Größte Ereignisse: &quot;Krieg&quot; &amp; Frieden')
expect(german.notes).toContainEqual({ path: 'params.introPage.title', message: expect.stringMatching(/^stores &, ", < and > as entities/) })

// Markup there is shown as written, as the editor's box would have stored it: noted, not refused.
const marked = await build({ ...META, type: 'H5P.QuestionSet', params: {
introPage: { showIntroPage: true, title: '<strong>WW2</strong>' },
questions: [{ library: 'H5P.TrueFalse', params: { question: 'Yes?', correct: 'true' } }]
} })
expect(marked.content.introPage.title).toBe('&lt;strong&gt;WW2&lt;/strong&gt;')
expect(marked.notes).toContainEqual({ path: 'params.introPage.title', message: 'is plain text, so its tags are shown as written, not as markup' })
})

it('lists every problem a source has at once, each by its path, and writes nothing', async () => {
expect(await problems({ ...META, type: 'H5P.Nope', params: {} })).toEqual(['type: the library pack has no content type H5P.Nope'])
// A version is saved content's, which checkContent takes; a build plays the pack's newest (#162).
Expand Down
27 changes: 26 additions & 1 deletion packages/author/tests/text.test.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest'
import { allowedTags, blockTag, escapeText, markupText, plainText } from '../lib/text.mjs'
import { allowedTags, blockTag, editorWidget, escapeText, markupText, plainText, takesMarkup } from '../lib/text.mjs'
import { translateDefaults, translationFiles } from '../lib/translate.mjs'

describe('text as H5P stores it', () => {
Expand All @@ -18,6 +18,31 @@ describe('text as H5P stores it', () => {
expect(blockTag({ tags: ['h2', 'strong'] })).toBe('p')
})

it('keeps an apostrophe and umlauts as typed in a plain box the content check filters', () => {
const notes: string[] = []
expect(plainText("Germany's Größe", (message) => notes.push(message), { filtered: true })).toBe("Germany's Größe")
expect(notes).toEqual([])
expect(plainText('Q&A "now" < 5 &amp; AT&T', (message) => notes.push(message), { filtered: true })).toBe('Q&amp;A &quot;now&quot; &lt; 5 &amp; AT&amp;T')
expect(notes).toHaveLength(1)
// A field the check escapes whole keeps the editor's escaping, umlauts as they are.
expect(plainText("Germany's Größe", () => {})).toBe('Germany&#39;s Größe')
})

it('takes markup only where the editor gives the field its rich-text widget', () => {
// Question Set 1.21's intro title: tags for the content check, a plain text box in the editor.
expect(takesMarkup({ type: 'text', tags: ['sub', 'sup', 'strong', 'em', 'code'] })).toBe(false)
expect(takesMarkup({ type: 'text', widget: 'textarea', tags: ['strong'] })).toBe(false)
expect(takesMarkup({ type: 'text', widget: 'html', tags: ['strong'] })).toBe(true)
// Speak the Words Set's introduction: a ShowWhen that draws the rich-text widget inside it.
const shown = { type: 'text', widget: 'showWhen', showWhen: { widget: 'html', rules: [] }, tags: ['strong'] }
expect(editorWidget(shown)).toBe('html')
expect(takesMarkup(shown)).toBe(true)
expect(editorWidget({ type: 'text', widget: 'showWhen', showWhen: { rules: [] } })).toBe('text')
expect(takesMarkup({ type: 'text', widget: 'showWhen', showWhen: { rules: [] }, tags: ['strong'] })).toBe(false)
// A rich-text field that lists no tags is escaped whole by the content check.
expect(takesMarkup({ type: 'text', widget: 'html' })).toBe(false)
})

it('wraps plain text written into a markup field as the editor wraps what an author types', () => {
const report = () => { throw new Error('nothing to report') }
expect(markupText('One line\nand the next\n\nA second: x < 5 & more', { tags: [], enterMode: 'p' }, report)).toBe(
Expand Down
Loading
Loading