Skip to content

perf(ruby): remove two superlinear costs from the RuboCop pass - #36

Merged
amritk merged 2 commits into
mainfrom
claude/ruby-performance-enhancements-n6cyld
Sep 9, 2026
Merged

amritk merged 2 commits into
mainfrom
claude/ruby-performance-enhancements-n6cyld

Conversation

@amritk

@amritk amritk commented Sep 9, 2026

Copy link
Copy Markdown
Member

What & why

Two performance patches for the RuboCop pass, applied at boot the way the existing syntax_tree patches are, so the artifact stays stock. Neither changes a byte of output; both change what a gem costs.

  1. The token sort, on every file with a heredoc in it. rubocop-ast's ProcessedSource#sorted_tokens falls back to a stable sort keyed on a two-element Array, [begin_pos, index], whenever the token list arrives out of position order — which any heredoc causes. sort_by compares Integers inline and Arrays through a method call into Array#<=>: ~150 ms on a 49 KB file with 4,738 tokens. The patch folds the pair into one Integer, begin_pos * count + index, which orders identically and compares as an Integer: ~9 ms. Over a 545 KB profile of 60 real files the sort had been 16.5% of everything the formatter did. (rubocop-ast master has since made the same change, unreleased; the patch retires when the pin moves past 1.50.0.)

  2. The line table, on every file with a multi-byte character in it. The same bug the 0.6.2 syntax_tree patch fixed, one layer down. parser's Buffer#line_begins walks the source with String#index("\n", from), which counts characters from the start on a non-ASCII string, so the table is O(size × lines) to build — and RuboCop rebuilds it on every correction round. On a 589 KB file with 17,401 lines and 729 carrying a multi-byte character it took 9.0 s per round. The patch walks the source once with each_line; the same table builds in ~25 ms. An ASCII source keeps the gem's own loop.

Node, one process, gems' own methods restored for the first run before after
206 files, 2.0 MB (rubygems, bundler, stdlib; 135 with a heredoc, 4 non-ASCII) 120.5 s 79.2 s
the 589 KB file alone 52.1 s 27.4 s

Which packages

packages/ruby

Exactness

Output is unchanged, and three things say so:

  • src/rubocop-perf-patch.test.ts asserts the patched sort returns the tokens object for object in the order the gem's own expression puts them, on five heredoc shapes that each reach the sort; and that the patched line table matches the gem's own loop entry for entry, sentinel included, on thirteen shapes of source (CRLF endings and multi-byte characters among them). A further test asserts both methods are the patched ones, so a patch that silently stops being applied fails rather than passes.
  • test/rubocop-conformance.test.ts gains one sample per patch, run through the real rubocop binary on a native Ruby with the pinned gems. It ran here (not skipped) and passes.
  • bun run ruby:bench --only corpus over the 206-file corpus above, snapshotted before and after and --compared: 206 of 206 hashes the same.

How it was validated

  • bun run check
  • bun run types:check
  • bun run test
  • bun run test:node
  • Rebuilt the wasm artifact — not needed; the patches are applied at boot and the artifact is untouched

Changeset

.changeset/quick-heredoc-token-sort.md, a patch bump for @scalar/ruby-fmt.

Two methods the Layout cops read through are replaced at boot, the way the
syntax_tree patches already are, so the artifact stays stock:

- rubocop-ast's ProcessedSource#sorted_tokens sorts on a [begin_pos, index]
  Array key, which compares through Array#<=> on every step. Any file with
  a heredoc takes that branch. The key is folded into one Integer,
  begin_pos * count + index, which orders identically and compares inline:
  ~150 ms -> ~9 ms on a 49 KB file with 4,738 tokens.

- parser's Buffer#line_begins walks the source with String#index(from),
  which counts characters from the start on a multi-byte string, so the
  line table costs O(size x lines) and RuboCop rebuilds it every round.
  A non-ASCII source is walked once with each_line instead: 9.0 s -> 25 ms
  on a 589 KB file, 52.1 s -> 27.4 s for the whole format.

Output is unchanged: the new tests assert each replacement answers exactly
as the gem's own expression on sources shaped to reach it, the native
RuboCop conformance test gains a sample per patch, and a 206-file corpus
hashes identically before and after while going from 120.5 s to 79.2 s
under Node.
@amritk amritk changed the title Optimize RuboCop token sort and line table on multi-byte sources perf(ruby): remove two superlinear costs from the RuboCop pass Sep 9, 2026
rubocop-ast's master already carries the token-sort fold, unreleased as
of 1.50.0, so that patch retires with the next release. parser's master
still walks line_begins with String#index as of 3.3.12.0, so that one
has no retirement date and is the fix to propose upstream.
@amritk
amritk merged commit de3bcf7 into main Sep 9, 2026
8 checks passed
@amritk
amritk deleted the claude/ruby-performance-enhancements-n6cyld branch September 9, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants