Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion packages/host/app/lib/matrix-classes/message.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { TrackedArray } from 'tracked-built-ins';
import { escapeHtmlOutsideCodeBlocks } from '@cardstack/runtime-common/helpers/html';
import {
markdownToHtml,
splitCodePatchFencesGluedToProse,
widenFencesAroundCodePatches,
} from '@cardstack/runtime-common/marked-sync';

Expand Down Expand Up @@ -233,7 +234,11 @@ export class Message implements RoomMessageInterface {
return this.body;
}
return markdownToHtml(
widenFencesAroundCodePatches(escapeHtmlOutsideCodeBlocks(this.body)!),
widenFencesAroundCodePatches(
splitCodePatchFencesGluedToProse(
escapeHtmlOutsideCodeBlocks(this.body)!,
),
),
{
sanitize: false,
escapeHtmlInCodeBlocks: true,
Expand Down
68 changes: 67 additions & 1 deletion packages/host/tests/unit/marked-sync-test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { escapeHtmlOutsideCodeBlocks } from '@cardstack/runtime-common/helpers/h
import {
markedSync,
markdownToHtml,
splitCodePatchFencesGluedToProse,
widenFencesAroundCodePatches,
} from '@cardstack/runtime-common/marked-sync';

Expand All @@ -20,7 +21,9 @@ import { parseHtmlContent } from '@cardstack/host/lib/formatted-message/utils';
// the applier matches it against the target file.
function renderBodyToCodeData(body: string) {
let html = markdownToHtml(
widenFencesAroundCodePatches(escapeHtmlOutsideCodeBlocks(body)!),
widenFencesAroundCodePatches(
splitCodePatchFencesGluedToProse(escapeHtmlOutsideCodeBlocks(body)!),
),
{
sanitize: false,
escapeHtmlInCodeBlocks: true,
Expand Down Expand Up @@ -67,6 +70,69 @@ ${REPLACE_MARKER}
`;

module('Unit | marked-sync', function () {
test('a patch whose opening fence ends a prose line still renders as a code block', function (assert) {
// The renderer opens a fence only at the start of a line, so a patch that
// begins "Let's write the block!```json" is prose to it: no code block,
// nothing applied, and the bot waits for a result that never comes.
let body = `I'll create the theme now. Let's write the block!\`\`\`json
https://example.com/realm/moon-theme.json (new)
${SEARCH_MARKER}
${SEPARATOR_MARKER}
{ "data": { "type": "card" } }
${REPLACE_MARKER}
\`\`\`
`;
let blocks = renderBodyToCodeData(body);

assert.deepEqual(
blocks.map((b) => b.fileUrl),
['https://example.com/realm/moon-theme.json'],
'the patch is found with its url on the first line',
);
assert.true(
blocks[0].searchReplaceBlock!.includes('{ "data": { "type": "card" } }'),
'the file content survives',
);
});

test('splitCodePatchFencesGluedToProse moves only the fence to its own line', function (assert) {
let body = `Let's write the block!\`\`\`json
https://example.com/realm/a.json (new)
${SEARCH_MARKER}
${SEPARATOR_MARKER}
{}
${REPLACE_MARKER}
\`\`\`
`;
assert.strictEqual(
splitCodePatchFencesGluedToProse(body),
`Let's write the block!\n\`\`\`json\n` +
body.split('\n').slice(1).join('\n'),
);
});

test('splitCodePatchFencesGluedToProse leaves prose that ends in backticks alone', function (assert) {
let inlineCode = 'Use the helper ```\nnot a patch\nsome more text\n';
assert.strictEqual(
splitCodePatchFencesGluedToProse(inlineCode),
inlineCode,
);

let properFence = `\`\`\`json
https://example.com/realm/a.json (new)
${SEARCH_MARKER}
${SEPARATOR_MARKER}
{}
${REPLACE_MARKER}
\`\`\`
`;
assert.strictEqual(
splitCodePatchFencesGluedToProse(properFence),
properFence,
'a fence already on its own line is unchanged',
);
});

test('a patch that writes markdown with fenced code inside renders as one block, and the patch after it keeps its url', function (assert) {
// Without widening, the first bare ``` inside the plan closes the patch's
// fence; the fence meant to close the patch then opens a block that
Expand Down
29 changes: 29 additions & 0 deletions packages/runtime-common/marked-sync.ts
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,35 @@ function normalizeDecorativeBullets(markdown: string): string {
// partially received plan file already renders as one block.
const FILE_URL_LINE_PATTERN = /^\s*https?:\/\/\S+(\s*\(\s*new\s*\))?\s*\r?$/;

// A fence opens a code block only at the start of a line. A model that ends a
// sentence and starts the patch on the same line — "Let's write the block!```json"
// — has written a patch the renderer reads as prose: the url, the markers and
// the file content collapse into one paragraph, the host finds no code block
// and applies nothing, and the bot, which counts patches by their markers,
// waits for a result that never comes. The block itself is correct; only the
// line break before the fence is missing. Put it back when the two lines after
// the fence are a file url and the SEARCH marker, which is what makes this a
// patch rather than prose that happens to end in backticks.
const FENCE_GLUED_TO_PROSE_PATTERN = /^(.*\S)(`{3,}\w*)\s*(\r?)$/;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude Code 🤖] Two edges in this pattern, both non-blocking:

A glued fence longer than three backticks splits in the wrong place. (.*\S) is greedy, so on Writing now!\```jsonthe prose group takes the first backtick: it yields proseWriting now!`and fence```json`. The prose line then ends in a stray backtick that opens inline code across the rest of the paragraph, and the opener is three backticks where the model wrote four.

(\r?) can never capture. The preceding \s* is greedy and consumes the \r first, so on CRLF input it is always '' and both produced lines lose their carriage return. Harmless to the render, but it reads as preserving CRLF and doesn't.

Both go away with a lazy prose group that can't end on a backtick, plus a horizontal-whitespace class so the \r survives to its group:

Suggested change
const FENCE_GLUED_TO_PROSE_PATTERN = /^(.*\S)(`{3,}\w*)\s*(\r?)$/;
const FENCE_GLUED_TO_PROSE_PATTERN = /^(.*?[^\s`])(`{3,}\w*)[ \t]*(\r?)$/;

Checked against both inputs: Go!\``json\r["Go!", "```json", "\r"], and the four-backtick line → ["Writing now!", "````json", ""]`.


Generated by Claude Code


export function splitCodePatchFencesGluedToProse(markdown: string): string {
let lines = markdown.split('\n');
for (let i = 0; i + 2 < lines.length; i++) {
let glued = lines[i].match(FENCE_GLUED_TO_PROSE_PATTERN);
if (
glued &&
!CODE_FENCE_PATTERN.test(lines[i]) &&
FILE_URL_LINE_PATTERN.test(lines[i + 1]) &&
SEARCH_MARKER_PATTERN.test(lines[i + 2])
Comment on lines +174 to +179

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid rewriting glued fences inside existing code blocks

When a valid outer patch—such as a Markdown or documentation patch—contains a literal example with prose glued to a fence followed by a file URL and SEARCH marker, this whole-document scan also matches inside that existing fenced block. If the example is in SEARCH content, the inserted newline prevents the patch from matching the target; in REPLACE content or a new file, it silently changes the requested file contents. Track fenced-block context and only repair top-level prose.

Useful? React with 👍 / 👎.

) {
let [, prose, fence, cr] = glued;
lines.splice(i, 1, `${prose}${cr}`, `${fence}${cr}`);
i++;
}
}
return lines.join('\n');
Comment on lines +173 to +186

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude Code 🤖] This scan carries no fenced-block context, so it rewrites patch content too — the bot flagged it and it is reachable. A patch that writes a document about the patch format carries the anti-example inside its own halves:

```markdown
https://example.com/realm/skill.md
╔══ SEARCH ══╗
╠══╣
Wrong — the fence must start a line:

Here it is!```json
https://example.com/realm/a.json (new)
╔══ SEARCH ══╗
╠══╣
{}
╚══ REPLACE ══╝
```

╚══ REPLACE ══╝
```

I ran that through splitCodePatchFencesGluedToProse: the inner Here it is!```json is split into two lines. In a REPLACE half that silently writes a file the model did not ask for; in a SEARCH half the extra newline makes the patch stop matching the target and the write fails.

normalizeDecorativeBullets in this file already carries the fence tracker this needs, for the same reason — fenced content has to survive verbatim. Tracking inFence here and splitting only at top level covers it; when a split does fire, the fence line it produces opens a block, so the state should be set from it.

Regression, introduced by this pass. Not a merge blocker on its own, but small enough that I'd fix it here rather than leave it as a follow-up.


Generated by Claude Code

}

export function widenFencesAroundCodePatches(markdown: string): string {
let lines = markdown.split('\n');
let i = 0;
Expand Down
Loading