Skip to content

fix(1/2): deterministic fenced-JSON extraction — linear worst case (CodeQL #1/#2) - #466

Closed
Alajmah wants to merge 2 commits into
masterfrom
security/fence-scanner-deterministic
Closed

Alajmah wants to merge 2 commits into
masterfrom
security/fence-scanner-deterministic

Conversation

@Alajmah

@Alajmah Alajmah commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Unit 2 of the authorized #425 remediation sequence (#5 → #1+#2 → #3 → #4). Remediation of CodeQL js/polynomial-redos #1 and #2 (reviewSchema.js:273/284) under the corrected acceptance criterion: a genuinely bounded extraction strategy, not another regex. Merge authority reserved; returned at exact head for clearance.

The defect

The fence-extraction regex pair — ```(?:json)?\s*\n?([\s\S]*?)\n?\s*``` in single-match (line 273) and matchAll (line 284) form — has an ambiguous whitespace bridge: the greedy \s* (which matches newlines) interacting with the following \n? lets the engine re-split long whitespace runs while the lazy body expands. The maintainer's Node 22 microbenchmark reproduced 2.06 s for one unclosed fence + 2,000 spaces; matchAll multiplies per fence. Input is the review LLM's completion (max_tokens: 32768, prompt-injection-echoable), and extraction runs on the shared worker event loop — pathological output stalls all 14 workers plus /health.

The correction (per the corrected criterion)

New scanFencedBlocks(): deterministic indexOf("```") scanning with monotonic cursors — fences pair sequentially (open runs to the next fence; scanning resumes after the close), every character is visited a bounded number of times, worst-case linear regardless of whitespace shape. No regex remains in the fence strategies.

Semantics preserved from the regex pair (each pinned by test):

  • Sequential pairing identical to matchAll's continuation-after-match semantics (odd fence counts → floor(n/2) blocks).
  • An immediate json label is stripped from the body (the (?:json)? branch, including its greedy following-whitespace run; JSON content never begins with the letters json, so the strip cannot destroy a parseable body — and the spaces-before-label case ``` json is left unstripped, matching the regex's non-match of the label there).
  • Unclosed trailing fence yields no block.
  • Caller-side .trim() before JSON.parse unchanged; strategy identifiers fenced / fenced_multi unchanged; strategies 2→2b precedence (first block, then last-to-first) unchanged.

Regressions (packages/rules/tests/review-schema-fence-scanner.test.js, 22 tests)

Equivalence — the replaced regexes are reproduced verbatim in-tree, and a 13-member corpus (labeled fences, label-without-newline, label-not-immediate, multi-fence both orders, adjacency, unclosed, mid-line, empty body) asserts identical extraction outcomes (strategy + parsed json) between old-regex and new-scanner code paths. Pairing and label-strip units pinned directly.

Boundedness — the maintainer-specified adversarial shapes, each asserted under a 500 ms wall-clock bound the old regex would blow through at these sizes:

  • unclosed fence + 200,000 spaces (the reproduced 2.06 s case, scaled 100×)
  • unclosed fence + 200,000 newlines (CodeQL's stated attack shape)
  • 30,000 repeated opening fences without closures
  • 5,000-fence valid multi-fence response (last fence valid → fenced_multi, full parse)
  • 150,000-space run inside a valid fence (extraction still succeeds — equivalence at scale)

Verification

  • New suite 22/22; pre-existing reviewSchema suite (60 tests) unchanged and green — behavior preservation pinned by the original tests, not just the new corpus.
  • Full @gitwire/rules suite: 328/328.
  • extractReviewJSON signature, strategies, and package public API unchanged (no new package export; the scanner is exported from the module for tests, mirroring reviewSchema.js's existing export style).

… YAML and shell layers

CodeQL js/incomplete-sanitization #5 remediation (flakyTestService.js:220):
testFilter escaped only double quotes. A test name containing a backslash
(specifically \") defeats that escaping in the generated Actions command —
the doubled backslash collapses and the quote closes the shell argument
early, executing the remainder as shell syntax in the runner. Dollar signs
and backticks also expand inside double quotes.

New boundary sanitizer sanitizeTestIdentifier(), applied uniformly at both
interpolation sites (filter and comments):
- YAML layer: all C0 controls + DEL become spaces — the only characters
  that can escape a command:| block scalar or a trailing comment line and
  inject workflow structure
- shell layer: backslash FIRST, then double quote, dollar sign, backtick

buildQuarantineWorkflow now exported for testing.

Regressions (flaky-quarantine-escaping.test.js, 13 tests): real js-yaml
parse of the generated document with structure assertions (one job, four
steps, injected runs-on/on: keys survive only as data); POSIX double-quote
decoder proving the closing quote is the intended one (rest === 2>&1) for
crafted \", $(), backtick, ${IFS}, newline and CR payloads; benign names
byte-identical (equivalent quarantine behavior).

Signed-off-by: Alajmah <55954113+Alajmah@users.noreply.github.com>
…nce regexes

CodeQL js/polynomial-redos #1/#2 remediation (reviewSchema.js:273/284). The
fence regex pair's greedy \s* (matches newlines) interacting with the
following \n? made the whitespace bridge ambiguous: O(n^2) per fence
occurrence on unclosed-fence + long-whitespace inputs, multiplied by
matchAll per fence. Input is the review LLM completion (max_tokens 32768) —
reachable via prompt-injection echo — and extraction runs on the shared
worker event loop, so pathological output blocked all 14 workers plus /health.

Replacement: scanFencedBlocks() — monotonic indexOf pairing of opening and
closing fences, linear worst case regardless of whitespace shape. Semantics
preserved from the regex pair: sequential pairing (matchAll continuation),
immediate-json label stripping (the (?:json)? branch; JSON never starts
with the letters json), unclosed trailing fence yields no block, caller-side
trim unchanged, strategy identifiers fenced/fenced_multi unchanged.

Regressions (review-schema-fence-scanner.test.js, 22 tests):
- EQUIVALENCE: old regexes reproduced verbatim in-tree; 13-member corpus
  (labels, label-not-immediate, multi-fence, adjacency, unclosed,
  mid-line) asserts identical strategy+json outcomes
- BOUNDEDNESS (maintainer-specified shapes, all under 500ms):
  unclosed fence + 200k spaces (the maintainer's reproduced 2.06s case,
  scaled), + 200k newlines (CodeQL's stated shape), 30k repeated opening
  fences, 5k-fence valid multi-fence response, 150k-space run inside a
  valid fence
- Pre-existing reviewSchema suite (60 tests) unchanged and green; full
  rules suite 328/328.

Signed-off-by: Alajmah <55954113+Alajmah@users.noreply.github.com>
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-20T23:38:30.822520Z
We couldn’t complete this review. Comment @codex review to try again.
0522dd1 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@Alajmah

Alajmah commented Nov 21, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #469: this branch was accidentally stacked on the #465 commit (two units in the diff) instead of branching from master. #469 is the same unit as a single commit on master. Its GitWire review also errored on provider output (ai_reviews 1443: 'JSON extraction failed — empty_input'), so nothing is lost.

@Alajmah Alajmah closed this Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant