rustdoc: Pre-calculate traits that are in scope for doc links

[petrochenkov]

This eliminates one more late use of resolver (part of #83761).
At early doc link resolution time we go through parent modules of items from the current crate, reexports of items from other crates, trait items, and impl items collected by collect-intra-doc-links pass, determine traits that are in scope in each such module, and put those traits into a map used by later rustdoc passes.
r? @jyn514

[rust-highfive]

Some changes occurred in intra-doc-links.

cc @jyn514

Some changes occurred in src/tools/clippy.

cc @rust-lang/clippy

[jyn514]

@bors try @rust-timer queue

Since encoding and decoding are performance sensitive.

[rust-timer]

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

[bors]

⌛ Trying commit 1514df2159ed22849eaddfd75c5e57cf801de6d4 with merge 2aadece0d9989be6855d4e96bac0f4ec08f23c91...

[petrochenkov]

Blocked on #88677 and #88872.

[mati865]

Typo? #88679 is this PR.

[petrochenkov]

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

[bors]

⌛ Trying commit 1ed1f29e01aedafd3a4dcb253de4be5da9a9eade with merge eec63217b0c2bd2bd82596fe0f8f6ccc5b9cc1a0...

[bors]

☀️ Try build successful - checks-actions
Build commit: 25308acc28ea882d78437b7fe14d09b78b93277b (25308acc28ea882d78437b7fe14d09b78b93277b)

[rust-timer]

Queued 25308acc28ea882d78437b7fe14d09b78b93277b with parent e7825f2, future comparison URL.

[rust-timer]

Finished benchmarking commit (25308acc28ea882d78437b7fe14d09b78b93277b): comparison url.

Summary: This change led to large relevant regressions 😿 in compiler performance.

• Large regression in instruction counts (up to 4.5% on full builds of cranelift-codegen doc)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR led to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

[petrochenkov]

Would it be possible to save the results of traits_in_scope in metadata or would it be too big?

This #88679 (comment) is the result of implementing that suggestion.
So it's possible to significantly reduce the overhead this way.

I'll save the optimization on a branch (https://github.com/petrochenkov/rust/tree/doctrscope4), but I don't want to land it now, because this PR is an intermediate step and I want to finish the whole resolver cloning removal before starting to encode more data for rustdoc, maybe we'll have different bottlenecks and need a different set of data in the end.

[camelid]

I still feel a bit uncomfortable with landing such a large regression, but I won't block this since I understand that it's an important refactoring. I'll let Guillaume take it from here.

[GuillaumeGomez]

@petrochenkov Let's go with this then. Can you open an issue with the next steps please (with explanations too, it'd be very much appreciated!). Once done, r=me. Thanks a lot for working on this!

[petrochenkov]

@GuillaumeGomez
#83761 is the issue, #83761 (comment) is the next step.

@bors r=GuillaumeGomez

[bors]

📌 Commit 00ba815 has been approved by GuillaumeGomez

[bors]

⌛ Testing commit 00ba815 with merge 788b1fe...

[bors]

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing 788b1fe to master...

[rust-timer]

Finished benchmarking commit (788b1fe): comparison url.

Summary: This benchmark run shows 43 relevant regressions 😿 to instruction counts.

• Average relevant regression: 5.4%
• Largest regression in instruction counts: 9.1% on full builds of regression-31157 doc

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Next Steps: If you can justify the regressions found in this perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please open an issue or create a new PR that fixes the regressions, add a comment linking to the newly created issue or PR, and then add the perf-regression-triaged label to this PR.

@rustbot label: +perf-regression

[trevyn]

@petrochenkov FYI, this seems to have broken one of my nightly CI runs, not sure if that's expected:

https://github.com/trevyn/turbosql/runs/4960657119?check_suite_focus=true

 Documenting turbosql-impl v0.4.0 (/home/runner/work/turbosql/turbosql/turbosql-impl)
Error: thread 'rustc' panicked at 'no entry found for key', src/librustdoc/passes/collect_intra_doc_links.rs:929:16
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

error: Unrecognized option: 'crate-version'

error: could not document `turbosql-impl`

[petrochenkov]

@trevyn
Could you make a minimized self-contained reproduction?
I'll look at this tomorrow, and having such a reproduction would make it much faster.

[trevyn]

Ok, apparently this is enough to trigger it on my side, not sure I feel like minimizing rusqlite right now:

Cargo.toml:

[package]
edition = "2018"
name = "test"
version = "0.1.0"

[dependencies]
rusqlite = "0.26"

lib.rs:

use rusqlite;

[jyn514]

@petrochenkov #93428

[petrochenkov]

The panic in #88679 (comment) happens because an inherent impl from rusqlite (impl Connection from mod busy) gets inlined into the current crate.

How is that even possible?
If I remove the doc link to make the documentation process to succeed, then that impl is not even in the generated documentation.

[camelid]

I think it would happen if Connection was inlined. Is Connection inlined?

[petrochenkov]

Maybe it's inlined internally, but it doesn't end up in the generated doc.
The documented crate in #88679 (comment) is just a single private import (it doesn't reexport Connection in particular), nothing should be inlined in that case.

[camelid]

Weird. I have no idea why that impl is being inlined then.

[petrochenkov]

I didn't figure out why that specific impl was inlined, but there are legitimate cases where inherent impls are inlined correctly, but traits in scope are not collected for them:

// Dependency crate

#[derive(Clone)]
pub struct PublicStruct;

mod inner {
    use super::PublicStruct;

    impl PublicStruct {
        /// [PublicStruct::clone]
        pub fn method() {}
    }
}

// Main crate

pub use dependency::PublicStruct;

[petrochenkov]

Both cases (#88679 (comment) and #88679 (comment)) are fixed in #93539.