Duplicated `c++` library in sysroot

[arnetheduck]

I'm looking through the sysroot in v34:

/wasi-sysroot/include$ ls
c++  wasm32-wasip1  wasm32-wasip1-threads  wasm32-wasip2  wasm32-wasip3
/wasi-sysroot/include$ du -shc c++ wasm32-wasip*/eh/c++ wasm32-wasip*/noeh/c++
0	c++
16M	wasm32-wasip1/eh/c++
16M	wasm32-wasip1-threads/eh/c++
16M	wasm32-wasip2/eh/c++
16M	wasm32-wasip3/eh/c++
16M	wasm32-wasip1/noeh/c++
16M	wasm32-wasip1-threads/noeh/c++
16M	wasm32-wasip2/noeh/c++
16M	wasm32-wasip3/noeh/c++
126M	total
/wasi-sysroot/include$ diff -r wasm32-wasip1/eh/c++ wasm32-wasip1/noeh/c++
/wasi-sysroot/include$ diff -r wasm32-wasip1/eh/c++ wasm32-wasip2/eh/c++

There are 6 seemingly identical copies of the C++ library that significantly contribute to its size - were they intended for the empty top-level c++ folder?

[alexcrichton]

This is currently intentional, yes, and it's an artifact of the various permutations of options and the affect on compiled artifact. The C++ standard library is first partitioned based on the target it's compiled for and then it's partitioned based on whether exception handling is implemented or not. That the headers are exactly the same is a coincidence for now but may not always be the case, so it's conservatively split. Basically it's easier to delegate the installation process to the library itself rather than having a bunch of WASI-specific logic.

One thing that might make sense though would be to do a post-processing pass over the installation artifact and replace separate files with at least hard links to one another if not symlinks on non-Windows platforms. That should help reduce the size of the download/install by a nontrivial amount and while still keeping the build process relatively the same.

[arnetheduck]

symlinks

In terms of prior art, llvm-mingw does something along these lines - there's a single generic-w64-mingw32/include folder that collects anything that is shared between the arch-specific <arch>-w64-mingw32 folders with symlinks pointing back to generic.

llvm-mingw uses <path-of-clang>/../normalized-triple/{include, lib} as the canonical "sysroot" during cross-compilation meaning that the setup "just works" without an explicit --sysroot parameter - very convenient.

As a happy coincidence, this also means that <path-of-clang>/../include is the shared canonical "generic" folder for "native" headers on windows.

wasm wouldn't have this luxury unfortunately so windows indeed needs a different solution.

I'm in the process of setting up a cross-compiler that can target both wasi (in its various incarnations) and mingw (with its numerous arches) as a single distribution - it ends up being gifted another identical copy of C++ from the llvm-mingw distribution.

libcxx itself maintains a division between a generic header folder and a target-specific, generated folder. Since this concept is enshrined in upstream, it should hopefully not go away any time soon.

[alexcrichton]

I've made an attempt at this in #656 drawing some inspiration hopefully from what you're describing. Perhaps not exactly applicable to your use case, but should cut down on the size here