Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 40 additions & 13 deletions scripts/lib/mdx-active-content.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ import {
isUnwrappedCodeBlock,
unwrapMdxCodeBlocks,
} from './mdx-code-blocks.mjs';
import { rewriteAfterUnwrap } from './mdx-preprocess.mjs';
import {
describeTarget,
disallowedScheme,
Expand Down Expand Up @@ -355,15 +356,29 @@ function mapUnwrappedLines(text, unwrapped) {
* Scans a Markdown/MDX body for anything outside the allowlists, and for
* images loaded from another host.
*
* The body is scanned as written and again as `unwrapMdxCodeBlocks()` leaves
* it, because that rewritten text is what the MDX compiler is handed. The
* per-node `mdx-code-block` check only sees the span mdast gave the node, and
* the fence grammars disagree about where it ends: CommonMark closes a
* three-backtick fence with any run of three or more, while Docusaurus
* requires exactly three and pairs the opener with a later bare fence. A
* payload between those two closers sits inside a `code` node the scan trusts.
* Scanning the unwrapped text closes that gap for any such disagreement. This
* mirrors `hasActiveMdx()` in ./mdx-escape.mjs.
* The body is scanned as written and again in each form Docusaurus's
* `preprocessContent()` produces, because that rewritten text, not the file,
* is what the MDX compiler is handed.
*
* `unwrapMdxCodeBlocks()` is the first of those rewrites. The per-node
* `mdx-code-block` check only sees the span mdast gave the node, and the fence
* grammars disagree about where it ends: CommonMark closes a three-backtick
* fence with any run of three or more, while Docusaurus requires exactly three
* and pairs the opener with a later bare fence. A payload between those two
* closers sits inside a `code` node the scan trusts. Scanning the unwrapped
* text closes that gap for any such disagreement. This mirrors
* `hasActiveMdx()` in ./mdx-escape.mjs.
*
* `rewriteAfterUnwrap()` (./mdx-preprocess.mjs) is the rest of the chain. The
* admonition rewrite in it inserts a bracket pair around a `:::note` title,
* which is the one character a run of literal text needs to its left to become
* a link: `:::note x](javascript:alert(1))` carries no link until Docusaurus
* supplies the `[`.
*
* Every form is scanned and the findings unioned rather than the last form
* being scanned alone, because a rewrite can also take capability away -- the
* heading-id escape inerts a `{...}` expression -- and a construct that is
* active in any form the compiler sees has to be reported.
*
* @param {string} markdown - Document body to scan.
* @param {{ mdastExtensions?: Array<object> }} [options] - Extra mdast
Expand All @@ -376,11 +391,23 @@ export function findActiveContent(markdown, { mdastExtensions = [] } = {}) {
const text = String(markdown ?? '');
const findings = scanSource(text, mdastExtensions);

// Both rewrites below operate line by line and neither adds nor removes a
// line, so a line of either rewritten form is the same line of `unwrapped`
// and maps back through the same table. The snippet is re-read from the
// source as well, so a finding quotes the text a reader can open the file
// to rather than the rewritten line it was spotted in.
const sourceLines = text.split('\n');
const unwrapped = unwrapMdxCodeBlocks(text);
if (unwrapped !== text) {
const sourceLine = mapUnwrappedLines(text, unwrapped);
for (const finding of scanSource(unwrapped, mdastExtensions)) {
findings.push({ ...finding, line: sourceLine[finding.line - 1] });
const sourceLine = mapUnwrappedLines(text, unwrapped);
for (const form of new Set([unwrapped, rewriteAfterUnwrap(unwrapped)])) {
if (form === text) continue;
for (const finding of scanSource(form, mdastExtensions)) {
const line = sourceLine[finding.line - 1];
findings.push({
...finding,
line,
snippet: sourceLines[line - 1].trim().slice(0, 120),
});
}
}

Expand Down
68 changes: 68 additions & 0 deletions scripts/lib/mdx-preprocess.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/**
* The rewrites Docusaurus applies to a page before the MDX compiler sees it.
*
* Both MDX gates in this directory rest on the premise that the text they scan
* is the text that gets compiled. It is not: `preprocessContent()` in
* @docusaurus/mdx-loader runs a chain of string rewrites over every page
* first, and only their output reaches MDX. A construct that is inert on disk
* and active after a rewrite is therefore invisible to a scan of the file
* alone.
*
* `unwrapMdxCodeBlocks()` is one link in that chain and already has a module of
* its own (./mdx-code-blocks.mjs). This one covers the rest of it, in the order
* the loader applies them:
*
* 1. `escapeMarkdownHeadingIds()` backslash-escapes `{#id}` in an ATX heading.
* 2. `admonitionTitleToDirectiveLabel()` turns the legacy `:::note Title`
* syntax into the container-directive form `:::note[Title]`.
*
* The second is the one that creates capability rather than removing it. It
* inserts a `[` after the directive token and a `]` at end of line, and a
* Markdown link needs exactly a `[` to its left and a `](url)` to its right --
* so `:::note x](javascript:alert(1))`, which is literal text in the file, is
* parsed as a link to `javascript:alert(1)` once the `[` is inserted. Neither
* rewrite adds or removes a line, so a finding's line number in the rewritten
* text is its line number in the text that went in.
*
* As with ./mdx-code-blocks.mjs, the upstream functions are called rather than
* their patterns restated, so where a bracket lands, which quote and
* indentation prefixes are accepted and how many colons open a directive
* cannot drift from the installed version.
*
* The one thing deliberately not taken from upstream is the keyword list.
* Docusaurus rewrites only the admonition names it knows, and a site may add
* more through `markdown.admonitions`; those names live in
* @docusaurus/mdx-loader, which is not a direct dependency and exports no entry
* point for them. Passing a name pattern instead of a list means the rewrite is
* applied to every directive name. That scans a form Docusaurus does not
* produce for a name it does not know, which can only report a finding that
* would not have shipped -- an over-rejection, never a miss -- and it holds if
* the keyword list is ever extended upstream or by configuration.
*/

import {
admonitionTitleToDirectiveLabel,
escapeMarkdownHeadingIds,
} from '@docusaurus/utils';

/**
* Stands in for the admonition keyword list, matching any directive name.
*
* `admonitionTitleToDirectiveLabel()` joins its keywords with `|` and drops
* them into a capture group, so a pattern here is interpolated into the same
* position a literal name would occupy.
*/
const ANY_DIRECTIVE_NAME = '[A-Za-z][A-Za-z0-9-]*';

/**
* Apply the preprocessor rewrites that follow `unwrapMdxCodeBlocks()`.
*
* @param {string} unwrapped - A document, already unwrapped.
* @returns {string} The text the MDX compiler is handed. Line count is
* unchanged, so line numbers carry over.
*/
export function rewriteAfterUnwrap(unwrapped) {
return admonitionTitleToDirectiveLabel(escapeMarkdownHeadingIds(unwrapped), [
ANY_DIRECTIVE_NAME,
]);
}
81 changes: 81 additions & 0 deletions tests/mdx-active-content.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -596,3 +596,84 @@ test('leaves ordinary fenced and inline code alone', () => {
].join('\n');
assert.deepEqual(reasons(body), []);
});

// Docusaurus hands the MDX compiler the output of preprocessContent(), not the
// file. unwrapMdxCodeBlocks() is only its first rewrite; the admonition
// rewrite that follows turns ":::note Title" into ":::note[Title]", and the
// bracket it inserts is the one character a run of literal text needs to its
// left to become a link. Each body below carries no link on disk and a link to
// a disallowed scheme once that bracket is supplied.
const PREPROCESSED = [
[
'admonition title rewrite forms a javascript: link',
':::note x](javascript:alert(document.domain))',
'disallowed URL scheme javascript:',
],
[
'admonition title rewrite forms a data:text/html link',
':::danger c](data:text/html;base64,PHNjcmlwdD4x)',
'disallowed URL scheme data:text/html',
],
[
'the rewrite applies inside a blockquote',
'> :::tip c](javascript:alert(1))',
'disallowed URL scheme javascript:',
],
[
'the rewrite applies to a four-colon directive',
'::::warning c](javascript:alert(1))',
'disallowed URL scheme javascript:',
],
[
'the rewrite is not limited to the upstream keyword list',
':::housekeeping c](javascript:alert(1))',
'disallowed URL scheme javascript:',
],
];

for (const [name, body, reason] of PREPROCESSED) {
test(`rejects a scheme that only appears after preprocessing: ${name}`, () => {
assert.deepEqual(reasons(body), [reason]);
});
}

test('reports a preprocessing-only finding at its true source line', () => {
// Neither rewrite after unwrapMdxCodeBlocks() adds or removes a line, so the
// line a reader opens the file to is the unwrapped line mapped back to the
// source -- here past a deleted mdx-code-block fence.
const body = [
'## Overview',
'',
'```mdx-code-block',
':::note x](javascript:alert(1))',
'```',
].join('\n');
assert.deepEqual(
findActiveContent(body).find(
(finding) => finding.reason === 'disallowed URL scheme javascript:',
),
{
line: 4,
reason: 'disallowed URL scheme javascript:',
snippet: ':::note x](javascript:alert(1))',
},
);
});

test('leaves an ordinary admonition and a fenced copy of the payload alone', () => {
// The rewrite is line-based and runs inside fenced code too, but a fence the
// unwrap step leaves in place still makes its body code to the MDX compiler,
// so neither body below publishes a link.
const body = [
':::note A normal title',
'',
'Body prose.',
'',
':::',
'',
'```',
':::note x](javascript:alert(1))',
'```',
].join('\n');
assert.deepEqual(reasons(body), []);
});
Loading