fix: LogParser 7x ARM64 regression — restore DFA for (?m)^#149
Merged
Conversation
adjustForAnchors() incorrectly routed (?m)^ patterns from UseDFA to UseNFA, claiming DFA can't verify multiline line anchors. The lazy DFA already handles (?m)^ correctly via StartByteMap/StartLineLF — identical to Rust regex approach. Verified with direct DFA tests. The downgrade caused 4 LangArena patterns (api_calls, post_requests, passwords, sessions) to use byte-by-byte NFA scan instead of DFA — catastrophic on ARM64 without SIMD (LogParser 2s -> 14s on M1). LangArena total: 2335ms -> 185ms (12.6x faster). Root cause: docs/dev/research/v01216-arm64-regression.md
…NFA candidate loop Literal extractor returned empty Seq on cross-product overflow (>250), killing all prefilter literals for patterns like (?i)(eval|system|exec|...). Now trims to 3-byte prefixes + dedup (Rust approach), marks inexact. Also guards NFA candidate loop with IsComplete() — incomplete prefilters cannot be used as correctness gates (would miss branches whose literals were truncated). NFA falls through to full scan instead. suspicious: UseNFA without prefilter (113ms) -> with FatTeddy (1ms). Stdlib compat: 38/38 PASS.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Benchmark ComparisonComparing Summary:
|
… safe selectLiteralStrategy blocked UseTeddy for any pattern with anchors, but adjustForAnchors already wraps prefilter with WrapLineAnchor for (?m)^. Added hasNonLineAnchors to allow UseTeddy when anchors are only (?m)^. http_methods on macOS ARM64: 89ms -> <1ms (restored to v0.12.14 level). Stdlib compat: 38/38 PASS.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix LogParser 7x regression on ARM64 reported by @kostya (#124).
(?m)^— lazy DFA already handles multiline line anchors correctly via StartByteMap/StartLineLF (identical to Rust). The downgrade forced 4 LangArena patterns to byte-by-byte NFA scan.(?i)alternation overflow — literal extractor returned empty Seq on overflow, killing all prefilter forsuspiciouspattern (549 NFA states). Now trims to 3-byte prefixes (Rust approach), marks inexact.IsComplete()— incomplete prefilters cannot be used as correctness gates (would miss truncated branches). NFA falls through to full scan.Results (LangArena simulation, 13 patterns)
Root cause analysis:
docs/dev/research/v01216-arm64-regression.mdTest plan
go test ./...— all passTestStdlibCompatibility— 38/38 PASS, 0 SKIPgolangci-lint run— 0 issues