Resolver: Wrap arenas in WorkerLocal

[LorrensP-2158466]

View all comments

In preperation of parallel import resolution #158845 this pr wraps the resolver arenas in WorkerLocal to ensure we don't need any synchronization to access an arena. This is common in the compiler.

Also, when building the resolution table of an external module, we first create a seperate table and fill that one, after which we replace the empty table with that new table. (Prep Work for #158845)
And a manual inline of define_extern.

r? @petrochenkov

[petrochenkov]

@bors try @rust-timer queue

[petrochenkov]

The PR description needs an update, it's not only about arenas.
@rustbot author

[rustbot]

Reminder, once the PR becomes ready for a review, use @rustbot ready.

[rust-bors]

☀️ Try build successful (CI)
Build commit: 8921b29 (8921b29e422cc22127146a787dda0186d080d041)
Base parent: 561ea2b (561ea2bfa3022226082e2db39d6ccb241e9e34bc)

[rust-timer]

Finished benchmarking commit (8921b29): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.3%]	2
Regressions ❌ (secondary)	0.3%	[0.1%, 0.6%]	16
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.4%]	6
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.1%]	6
All ❌✅ (primary)	-0.2%	[-0.5%, 0.3%]	8

Max RSS (memory usage)

Results (primary 1.9%, secondary 1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.9%	[1.9%, 1.9%]	1
Regressions ❌ (secondary)	6.2%	[6.2%, 6.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.7%	[-3.7%, -3.7%]	1
All ❌✅ (primary)	1.9%	[1.9%, 1.9%]	1

Cycles

Results (primary -3.0%, secondary -3.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-3.0%	[-3.0%, -3.0%]	1
Improvements ✅ (secondary)	-3.0%	[-3.0%, -3.0%]	1
All ❌✅ (primary)	-3.0%	[-3.0%, -3.0%]	1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 489.147s -> 488.199s (-0.19%)
Artifact size: 388.47 MiB -> 388.92 MiB (0.12%)

[petrochenkov]

Small regressions on two stress benchmarks from worker local arenas.
It's acceptable, and we'll need to integrate resolver into the parallel compiler sooner or later anyway.

[petrochenkov]

The build reduced graph changes are potential micro-optimizations (that's where the diesel improvements come from), and the worker local changes are pessimizations, ideally they'd be benchmarked separately.

[LorrensP-2158466]

Force-pushed the comments to keep it one commit. @rustbot ready

ideally they'd be benchmarked separately.

would you like 2 separate prs?

[petrochenkov]

would you like 2 separate prs?

Let's not complicate things and just merge.
@bors r+

[rust-bors]

📌 Commit 72e9ddc has been approved by petrochenkov

It is now in the queue for this repository.

[rust-bors]

☀️ Test successful - CI
Approved by: petrochenkov
Duration: 3h 9m 11s
Pushing 3b58636 to main...

[github-actions]

What is this?

This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing ad49852 (parent) -> 3b58636 (this PR)

Test differences

Show 3 test diffs

Stage 2

• [run-make] tests/run-make/compressed-debuginfo-zstd: ignore (ignored if LLVM wasn't build with zstd for ELF section compression or LLVM is not the default codegen backend) -> pass (J0)

Additionally, 2 doctest diffs were found. These are ignored, as they are noisy.

Job group index

• J0: x86_64-gnu-nopt

Test dashboard

Run

cargo run --manifest-path src/ci/citool/Cargo.toml -- \
    test-dashboard 3b58636b30eb364ac72aeaf03d46347084ed87d1 --output-dir test-dashboard

And then open test-dashboard/index.html in your browser to see an overview of all executed tests.

Job duration changes

1. x86_64-gnu-gcc-core-tests: 19m 4s -> 8m 40s (-54.5%)
2. x86_64-gnu-llvm-22-1: 1h 10m -> 42m 54s (-39.0%)
3. i686-gnu-nopt-2: 1h 40m -> 2h 19m (+38.5%)
4. x86_64-msvc-2: 1h 46m -> 2h 24m (+35.2%)
5. x86_64-msvc-ext2: 1h 21m -> 1h 47m (+32.8%)
6. pr-check-2: 49m 36s -> 33m 29s (-32.5%)
7. pr-check-1: 40m 50s -> 28m 1s (-31.4%)
8. x86_64-rust-for-linux: 1h 2m -> 43m 40s (-30.6%)
9. x86_64-gnu-aux: 3h -> 2h 9m (-28.3%)
10. test-various: 2h 26m -> 1h 46m (-27.4%)

How to interpret the job duration changes?

Job durations can vary a lot, based on the actual runner instance
that executed the job, system noise, invalidated caches, etc. The table above is provided
mostly for t-infra members, for simpler debugging of potential CI slow-downs.

[rust-timer]

Finished benchmarking commit (3b58636): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Our benchmarks found a performance regression caused by this PR.
This might be an actual regression, but it can also be just noise.

Next Steps:

• If the regression was expected or you think it can be justified,
  please write a comment with sufficient written justification, and add
  @rustbot label: +perf-regression-triaged to it, to mark the regression as triaged.
• If you think that you know of a way to resolve the regression, try to create
  a new PR with a fix for the regression.
• If you do not understand the regression or you think that it is just noise,
  you can ask the @rust-lang/wg-compiler-performance working group for help (members of this group
  were already notified of this PR).

@rustbot label: +perf-regression
cc @rust-lang/wg-compiler-performance

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.0%, 0.5%]	10
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.2%]	10
Improvements ✅ (secondary)	-0.2%	[-0.3%, -0.0%]	7
All ❌✅ (primary)	-0.4%	[-0.5%, -0.2%]	10

Max RSS (memory usage)

Results (primary -0.7%, secondary 1.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	2.3%	[2.3%, 2.3%]	1
Regressions ❌ (secondary)	1.2%	[0.9%, 1.6%]	2
Improvements ✅ (primary)	-3.8%	[-3.8%, -3.8%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.7%	[-3.8%, 2.3%]	2

Cycles

Results (primary -2.0%, secondary 1.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.3%	[2.4%, 11.0%]	9
Improvements ✅ (primary)	-2.0%	[-2.0%, -2.0%]	1
Improvements ✅ (secondary)	-4.3%	[-8.6%, -1.5%]	5
All ❌✅ (primary)	-2.0%	[-2.0%, -2.0%]	1

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 488.317s -> 490.631s (0.47%)
Artifact size: 389.82 MiB -> 389.34 MiB (-0.12%)