Upgrade to LLVM 11 (rc2)

[cuviper]

This builds on #73525 to try actually moving rust-lang/llvm-project to LLVM 11.

[rust-highfive]

@cuviper: no appropriate reviewer found, use r? to override

[rust-highfive]

⚠️ Warning ⚠️

• These commits modify submodules.

[cuviper]

r? @nikic

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion

[bors]

⌛ Trying commit 9e693e204b536e4ca493056004403623574515ee with merge 3850eb2b77cbad53a08dafa5cc8c355c61cfa7fc...

[bors]

💔 Test failed - checks-azure

[cuviper]

Sorry, I hadn't pushed the latest llvm commit to that branch.

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion

[bors]

⌛ Trying commit bcdd0dbb05fc69c475e1c6565449ece566f18df3 with merge 60407dcb5300e11b39bb7ecdb054f4ca12b21d06...

[cuviper]

For reference: llvm/llvm-project@master...cuviper:rustc/11.0-2020-06-19

[bors]

💔 Test failed - checks-azure

[cuviper]

Hmm, CPU_COUNT is assumed starting in llvm/llvm-project@041a355, but while sched_getaffinity has been around as long as stated there, CPU_COUNT was only added in glibc 2.6. Our CI on CentOS 5 only has glibc 2.5.

As it happens, I also have a branch where I'm updating that to CentOS 6 (see also #62516), but that will need some kind of signoff to raise the supported OS.

[nikic]

Hmm, CPU_COUNT is assumed starting in llvm/llvm-project@041a355, but while sched_getaffinity has been around as long as stated there, CPU_COUNT was only added in glibc 2.6. Our CI on CentOS 5 only has glibc 2.5.

For now, we can just revert that commit. We already have a few workarounds for CentOS 5...

As it happens, I also have a branch where I'm updating that to CentOS 6 (see also #62516), but that will need some kind of signoff to raise the supported OS.

Let's get that started then :) This is a recurring problem, every LLVM update breaks compatibility with CentOS 5 more.

[cuviper]

I reverted two CPU_COUNT commits, and now I have a clean docker build locally, so... 🤞

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion

[bors]

⌛ Trying commit 6f91e02651a42b5deec704e07f3540eeb6cce9af with merge 5ba9571d17e40ba72702b33e2d565f807eab0446...

[bors]

☀️ Try build successful - checks-azure
Build commit: 5ba9571d17e40ba72702b33e2d565f807eab0446 (5ba9571d17e40ba72702b33e2d565f807eab0446)

[rust-timer]

Queued 5ba9571d17e40ba72702b33e2d565f807eab0446 with parent 1033351, future comparison URL.

[rust-timer]

Finished benchmarking try commit (5ba9571d17e40ba72702b33e2d565f807eab0446): comparison url.

[nikic]

Huh, that's unexpected. Given that we have major check build regressions, I'm assuming this is not due to another LLVM compile-time regression, but an optimization regression that affects rustc very badly.

[cuviper]

That will be sad if the apparent opt time wins are just by doing less optimization, especially with noticeable effects. Do you have any tips for how to track down the difference? Tracing LLVM passes or something?

[cuviper]

I pushed rust-lang/llvm-project@45790d7 to comment out that assertion.

@bors r=nikic

[bors]

📌 Commit b450c0c has been approved by nikic

[bors]

⌛ Testing commit b450c0c with merge cf920393ef69e6340dcc19dc86cf547171560a33...

[rust-log-analyzer]

The job mingw-check of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

+ /scripts/validate-toolstate.sh
Cloning into 'rust-toolstate'...
<Nothing changed>
Traceback (most recent call last):
  File "../../src/tools/publish_toolstate.py", line 283, in <module>
    validate_maintainers(repo, github_token)
  File "../../src/tools/publish_toolstate.py", line 84, in validate_maintainers
    'Accept': 'application/vnd.github.hellcat-preview+json',
  File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden
  local time: Sat Aug 22 21:25:38 UTC 2020
  network time: Sat, 22 Aug 2020 21:25:38 GMT
== end clock drift check ==
== end clock drift check ==
##[error]Process completed with exit code 1.
Terminate orphan process: pid (2854) (node)
Terminate orphan process: pid (2882) (python)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @rust-lang/infra. (Feature Requests)

[bors]

💔 Test failed - checks-actions

[Aaron1011]

@bors retry

Spurious network issue.

[bors]

⌛ Testing commit b450c0c with merge 7ce71c3...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: nikic
Pushing 7ce71c3 to master...

[cuviper]

We're up to eleven! 🤘

[samanpa]

Can we have the release notes mention the in tree llvm version? Right now one has to keep track of PRs to know which llvm to use if one doesn’t use the in tree llvm.

[cuviper]

This PR is on the 1.47 milestone and labeled relnotes, so it should be mentioned. But also, external LLVM doesn't have to match here -- we currently support back to LLVM 8.

[ecstatic-morse]

Final perf results are in and show a significant and sustained improvement in wall time for optimized builds on most benchmarks. Thanks all!

[mati865]

@cuviper regarding rust-lang/llvm-project@08116dc, rust-lld apparently does use threadpool. I'll try to take care of that but I thought you might want to know it.

[cuviper]

@mati865 oh, ok -- is that at least falling back to a single thread as intended?

[mati865]

@cuviper i686 windows-gnu rust-lld still works with ThinLTO enabled so I think the answer is yes.

[tspiteri]

@cuviper It seems that #76042 is hitting an LLVM 11 blocker. Is it a good idea to have the current beta 1.47 depend on an unreleased LLVM which still has a few blockers?

[cuviper]

@tspiteri the compiler team explicitly approved the risk of landing pre-release: #73526 (comment)
Even after release, compilers may have bugs/regressions, so there's no perfect upgrade.

If an LLVM fix comes soon for your issue, we can backport it to our LLVM branch or pick that up in a rollup to the next 11-RC or final release. Otherwise, if we decide that your issue is a release-blocker for Rust 1.47, then this can be reverted from beta.