Intrinsic implementation prevents instruction shuffling

[krzikalla]

Just consider the generated code for the two foo variants:

https://godbolt.org/z/P7TMe4vax

The code using double interleaves multiplications for the first and second summand to achieve more ILP. The code using std::simd doesn't do this.
IMHO the reason for this is the usage of intrinsics in the std::simd implementation, which prevents such optimizations. This can have quite an impact on the acceleration you get from vectorization.
I have no good solution for this issue yet, I just want to raise the awareness here.
Maybe some annotations for intrinsics must be introduced, which can tell the compiler, that the annotated intrinsic is allowed to be optimized.

[mattkretz]

Hi Olaf, we should talk! I watched your CppCon talk and was cheering you on. 😁 The problems you solved are problems I looked into 15 years ago, but I'm still working on the basics for the C++ standard. I couldn't find your e-mail address yet. You can find mine in the header of basically every source file 😉.

On this issue:
Most of the simd implementation in libstdc++ doesn't use intrinsics but directly uses [[gnu::vector_size(N)]] types and their operators. Only where that's not possible does it fall back to using intrinsics/builtins.

This specific code example isn't a fair comparison because GCC emits a clone for the doubles where it doesn't pass a Foo reference but a double. With the noipa attribute the codegen is the same: https://godbolt.org/z/7557n7Txa

Also, as long as there are no dependencies, the CPU will reorder instructions. The instruction decoder windows size could be an issue, but as soon as there's a few things in the pipeline it shouldn't matter.

For what it's worth, when you find a missed optimization in GCC, report it on GCC's Bugzilla. GCC devs have already implemented a lot of optimizations that help simd types that way. (I have a queue of things still to report — which takes time to do properly).