Did you have a look on godbolt testcase that is mentioned in the comment? https://godbolt.org/z/7EWqrYq51

For reference, our current pre-release binaries can be compiled with gcc 11 on Linux, and gcc 13 on windows (x86)

I suggest benchmark on VNNI512 / AVXVNNI architectures also, given that the ASM benefits VNNI more.

@vondele I don't trust myself to read the godbolt assembly well enough to tell if it's optimal or not spilling registers. Which is why I ask for @Sopel97 opinion. I should also be able to test using gcc 11 (but on Windows) if it's useful to know. Are the compilers for the pre-release binaries the same as what fishtest uses?

@cj5716 I should be able to test VNNI512 but don't have access to AVXVNNI hardware. Where does the more benefit for VNNI information come from?

@vondele I don't trust myself to read the godbolt assembly well enough to tell if it's optimal or not spilling registers. Which is why I ask for @Sopel97 opinion. I should also be able to test using gcc 11 (but on Windows) if it's useful to know. Are the compilers for the pre-release binaries the same as what fishtest uses?

@cj5716 I should be able to test VNNI512 but don't have access to AVXVNNI hardware. Where does the more benefit for VNNI information come from?

There is VNNI specific ASM in the code that has fewer instructions, which is why I think benchmarking on VNNI is important.

I can't replicate any register spilling issues anymore, and I trust the test. Not sure what the representation for VNNI on fishtest is right now, probably could be benchmarked specifically here too.

@vondele I don't trust myself to read the godbolt assembly well enough to tell if it's optimal or not spilling registers. Which is why I ask for @Sopel97 opinion. I should also be able to test using gcc 11 (but on Windows) if it's useful to know. Are the compilers for the pre-release binaries the same as what fishtest uses?

fishtest uses what the users have installed, so it can be anything. For the pre-releases on github we have more control (yet certain limitations, not everything is available one every os).