perf(engine): memoize glob regex compilation with bounded cache

[iam-saiteja]

What Was Happening

During workspace file indexing and ripgrep search filtering, functions like pathPatternMatches, ripgrepGlobMatches, and pathPatternMightMatchDescendant repeatedly call globToRegExp(pattern, caseInsensitive).

Every single path check re-parsed the glob string and compiled a brand-new RegExp object from scratch. With hundreds of repository paths and multiple ignore or include patterns, this caused repeated regular expression compilation overhead on hot paths.

What Changed

1. Added Bounded Regex Cache (src/engine/utils/glob.ts)

   • Stored compiled RegExp objects in a Map<string, RegExp> with a limit of 1024 entries.
   • Added i: (case-insensitive) and s: (case-sensitive) prefixes to cache keys to prevent key collisions between different sensitivity modes.
   • Used native Map insertion-order eviction to safely evict the oldest entry when reaching capacity, preventing unbounded memory growth.
   • Kept regexes stateless (no g or y flags) so RegExp.prototype.test() has zero side effects across shared calls.

2. Added Real-Path Benchmark (benchmarks/glob-perf.bench.mjs)

   • Created a standalone benchmark script that reads real file paths directly from the repository.
   • Tests 37 realistic pattern rules (ignore rules, file extensions, nested directories, brace expansions).
   • Embeds the original unoptimized baseline logic directly in the script to compare performance and verify 100% output parity with node:assert/strict.

Commits

• c817949: benchmarks: add real-path glob pattern matching benchmark
  • Added benchmarks/glob-perf.bench.mjs to measure baseline matching performance across actual project paths.
• f670ca8: perf: memoize glob regex compilation with bounded cache
  • Added bounded regex caching in src/engine/utils/glob.ts.
  • Updated benchmarks/glob-perf.bench.mjs to benchmark baseline vs optimized logic side by side and assert zero output mismatches.

Performance Results (3.3 Million Pattern Evaluations)

Function	Baseline (Before)	Optimized (After)	Speedup	Time Reduction	Parity
pathPatternMatches	14,858.93 ms	5,598.65 ms	2.7x	62.3%	0 mismatches
ripgrepGlobMatches	17,054.90 ms	5,818.10 ms	2.9x	65.9%	0 mismatches
pathPatternMightMatchDescendant	35,521.84 ms	17,471.54 ms	2.0x	50.8%	0 mismatches
Total	67,435.67 ms (~67.4s)	28,888.30 ms (~28.9s)	2.3x	57.2%	100% Match

How To Reproduce & Verify

1. Build the TypeScript source:

   npm run build

2. Run the performance benchmark:

   node benchmarks/glob-perf.bench.mjs

3. Run the unit test suite:

   npm run test:unit

4. Run the linter:

   npm run lint