Repository navigation
fix(1/2): deterministic fenced-JSON extraction — linear worst case (CodeQL #1/#2) [v2 sibling] - #469
Conversation
…nce regexes CodeQL js/polynomial-redos #1/#2 remediation (reviewSchema.js:273/284). The fence regex pair's greedy \s* (matches newlines) interacting with the following \n? made the whitespace bridge ambiguous: O(n^2) per fence occurrence on unclosed-fence + long-whitespace inputs, multiplied by matchAll per fence. Input is the review LLM completion (max_tokens 32768) — reachable via prompt-injection echo — and extraction runs on the shared worker event loop, so pathological output blocked all 14 workers plus /health. Replacement: scanFencedBlocks() — monotonic indexOf pairing of opening and closing fences, linear worst case regardless of whitespace shape. Semantics preserved from the regex pair: sequential pairing (matchAll continuation), immediate-json label stripping (the (?:json)? branch; JSON never starts with the letters json), unclosed trailing fence yields no block, caller-side trim unchanged, strategy identifiers fenced/fenced_multi unchanged. Regressions (review-schema-fence-scanner.test.js, 22 tests): - EQUIVALENCE: old regexes reproduced verbatim in-tree; 13-member corpus (labels, label-not-immediate, multi-fence, adjacency, unclosed, mid-line) asserts identical strategy+json outcomes - BOUNDEDNESS (maintainer-specified shapes, all under 500ms): unclosed fence + 200k spaces (the maintainer's reproduced 2.06s case, scaled), + 200k newlines (CodeQL's stated shape), 30k repeated opening fences, 5k-fence valid multi-fence response, 150k-space run inside a valid fence - Pre-existing reviewSchema suite (60 tests) unchanged and green; full rules suite 328/328. Signed-off-by: Alajmah <55954113+Alajmah@users.noreply.github.com>
Codex Review Summary: ❌ FailedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
/gitwire run review |
|
|
There was a problem hiding this comment.
🤖 AI Code Review — 💬 Needs discussion
Confidence: low · Findings: 5
AI judgment: NEEDS DISCUSSION · Evidence: COMPLETE · Authority: advisory
Evidence complete · 2/2 changed files accounted for
Other findings (5)
- [low] Equivalence corpus rows with no old-pipeline outcome assert nothing
- [medium] No test actually enforces the headline 'linear worst case' claim
⚠️ unverified - [medium] Multi-fence documents: the deterministic selection rule is unpinned by tests
⚠️ unverified - [low] Equivalence corpus omits CRLF, wider-backtick, tilde, and info-suffix fence variants
⚠️ unverified - [low] Legacy pipeline retained for equivalence runs keeps the flagged quadratic behavior alive in CI
⚠️ unverified
🔮 Devil's Advocate (2 turns): 4 missed risks found · 1 upheld
GitWire AI Review Gate (bundle-driven v2) · Structured schema · Scope-validated · Devil's Advocate · AI recommendation — maintainers and repository policy retain authority
Finding details (not annotatable inline)
-
[MEDIUM] No test actually enforces the headline 'linear worst case' claim
The PR exists to fix CodeQL #1/#2 (super-linear complexity), but boundedness-style tests that feed one large input and assert completion cannot distinguish O(n) from O(n^2); a regressed quadratic scanner passes silently until CI times out flakily. The review never asked whether any test discriminates complexity classes (scaling assertions, operation counters, or an explicit input-size cap), so the fix's core guarantee is unprotected against future regression.
-
[MEDIUM] Multi-fence documents: the deterministic selection rule is unpinned by tests
The title says 'deterministic fenced-JSON extraction', implying candidate selection among multiple fences changed. The corpus shown contains only single-fence inputs. With no rows containing two or more JSON fences (e.g., a documented example fence plus the real payload), the tie-break rule (first, last, or largest matching block) is unpinned, and a future change could silently alter which block gets extracted while every existing test stays green.
-
[LOW] Equivalence corpus omits CRLF, wider-backtick, tilde, and info-suffix fence variants
Scanner rewrites most often diverge from the legacy pipeline on line-ending handling (CRLF), fences of more than three backticks, tilde fences, and trailing info after the label. These shapes are common in real reviews (Windows-authored content, pasted Markdown) and are exactly where index/label arithmetic regresses; the corpus covers mid-line text and spaces-before-label but not these, so 'equivalence' is unproven for them.
-
[LOW] Legacy pipeline retained for equivalence runs keeps the flagged quadratic behavior alive in CI
Equivalence testing requires executing the pre-fix implementation (embedded copy or imported from history) inside the test suite. That code is precisely what CodeQL flagged; feeding it large or adversarial corpus inputs reintroduces the pathological runtime into CI builds, and the duplicate implementation becomes a permanent fixture unless the review or PR notes define a deletion point once the migration window closes.
| if (oldOutcome === null) { | ||
| // Old code fell through to strategies 3-5; the new code must reach the | ||
| // SAME subsequent strategies. For corpus members designed to be decided | ||
| // at the fence stage, falling through means both sides continue — the |
There was a problem hiding this comment.
[LOW] Equivalence corpus rows with no old-pipeline outcome assert nothing
In assertFenceEquivalence(), when oldOutcome === null the function returns extracted without any expect() call, despite the inline comment claiming "the extraction result must then still agree, so compare full results". Consequently several corpus entries execute the new pipeline but verify no outcome at all: ' json\n{"e": 5}\n' (spaces-before-label — the row's own comment claims "both must fail fence parse identically", which is never checked), 'unclosed json body without end', and 'text before mid-line {"h": 8} ``` text after'. A future regression in fall-through behavior or label handling for these shapes would pass the suite. Fix: replace the silent return with explicit expectations (e.g., assert extracted.json/strategy against hand-computed values, or assert null for designed fall-through rows). Partially mitigated by the dedicated scanFencedBlocks label/pairing tests and the boundedness tests, so this is a gap rather than a correctness bug.
Review adjudication — all 5 findings (evidence-closed; none blocking; scanner source unchanged)F1 [medium] No test discriminates the linear-worst-case claim. Covered by construction: the boundedness assertions run at 200,000-character inputs with a 500 ms bound — sizes chosen precisely so a regressed polynomial scanner cannot pass them. The maintainer's reproduced baseline for the OLD regex was ~2.06 s at one unclosed fence + 2,000 spaces (n=2k); the same complexity at n=200k scales ~10⁴× → minutes of runtime — a hard, non-flaky failure against the 500 ms bound, not a silent pass. A t(n)-vs-t(2n) scaling assertion would add class-discrimination but also CI timing noise; recorded as optional test hardening, not a gap in the fix's protection. F2 [medium] Multi-fence selection rule unpinned. Premise false — the suite pins the selection rule in three places: corpus rows F3 [low] Corpus omits CRLF / wider-backtick / tilde / info-suffix fences. Valid gap, minor: these variants are untested though the implementations are expected to agree (tilde fences are recognized by neither; wider backtick runs pair identically at the first ``` in both; CRLF is consumed by F4 [low] Legacy pipeline keeps the flagged quadratic behavior alive in CI. Overstated: F5 [low] Vacuous corpus rows. Valid: rows where the old pipeline falls through return without asserting (later strategies decide identically on both sides — those code paths are untouched — but the rows assert nothing themselves). Folded into the corpus-extension cleanup. State: head |
Summary
Unit 2 of the authorized #425 remediation sequence (#5 → #1+#2 → #3 → #4). Remediation of CodeQL
js/polynomial-redos#1 and #2 (reviewSchema.js:273/284) under the corrected acceptance criterion: a genuinely bounded extraction strategy, not another regex. Merge authority reserved; returned at exact head for clearance.The defect
The fence-extraction regex pair —
```(?:json)?\s*\n?([\s\S]*?)\n?\s*```in single-match (line 273) andmatchAll(line 284) form — has an ambiguous whitespace bridge: the greedy\s*(which matches newlines) interacting with the following\n?lets the engine re-split long whitespace runs while the lazy body expands. The maintainer's Node 22 microbenchmark reproduced 2.06 s for one unclosed fence + 2,000 spaces;matchAllmultiplies per fence. Input is the review LLM's completion (max_tokens: 32768, prompt-injection-echoable), and extraction runs on the shared worker event loop — pathological output stalls all 14 workers plus/health.The correction (per the corrected criterion)
New
scanFencedBlocks(): deterministicindexOf("```")scanning with monotonic cursors — fences pair sequentially (open runs to the next fence; scanning resumes after the close), every character is visited a bounded number of times, worst-case linear regardless of whitespace shape. No regex remains in the fence strategies.Semantics preserved from the regex pair (each pinned by test):
matchAll's continuation-after-match semantics (odd fence counts → floor(n/2) blocks).jsonlabel is stripped from the body (the(?:json)?branch, including its greedy following-whitespace run; JSON content never begins with the lettersjson, so the strip cannot destroy a parseable body — and the spaces-before-label case``` jsonis left unstripped, matching the regex's non-match of the label there)..trim()beforeJSON.parseunchanged; strategy identifiersfenced/fenced_multiunchanged; strategies 2→2b precedence (first block, then last-to-first) unchanged.Regressions (
packages/rules/tests/review-schema-fence-scanner.test.js, 22 tests)Equivalence — the replaced regexes are reproduced verbatim in-tree, and a 13-member corpus (labeled fences, label-without-newline, label-not-immediate, multi-fence both orders, adjacency, unclosed, mid-line, empty body) asserts identical extraction outcomes (strategy + parsed json) between old-regex and new-scanner code paths. Pairing and label-strip units pinned directly.
Boundedness — the maintainer-specified adversarial shapes, each asserted under a 500 ms wall-clock bound the old regex would blow through at these sizes:
fenced_multi, full parse)Verification
@gitwire/rulessuite: 328/328.extractReviewJSONsignature, strategies, and package public API unchanged (no new package export; the scanner is exported from the module for tests, mirroringreviewSchema.js's existing export style).v2 note: replaces #466 — the original branch was accidentally stacked on the #465 commit, so its diff carried two units. This branch is a single commit on master (89d5e51) carrying only the fence-scanner unit; content identical (cherry-picked).