[CI] Setup compiler sometimes takes >30mins !

[AntoinePrv]

And it is not even bootstrapping 😅

I think apt-get in CI is pretty common so that's not the diet suspect. Maybe PPA ?
Or GitHub itself limiting our bandwidth?

Alternatively should we find compilers docker images ?

[AntoinePrv]

Other things to investigate as well to speed up CI:

• sccache: free rebuild for all test files that are unmodified (say the AVX2 file is modified, all the SSE build remain unchanged). Clever so we won't get a cache unless it matches perfectly.
• Github action path / path_ignore: Add a coarser level, we can simply SKIP some architecture (e.g. ARM) if all changes fall within another arch (e.g. x86), and keep it all in main just to be safe. Would need to reorganize the headers because they are not grouped by family (and maybe we can make all CI fire when we add a special label).

[Alex-PLACET]

We can switch to namespace.so runners as we did for xtensor, it's far more fast than github runners and have efficient cache.