Just a proposal, but I thought it was easier to discuss with a proposal as opposed to an issue. The add_compile_options($<$<CONFIG:Release>:-O2>) line adds the -O2 on Release builds, but the Release builds on CMake already pass to the compiled the -O3 flag. Passing also the -O2 options results in both options passed to the compiled, see for example (extracted from compile_commands.json):

{
  "directory": "/home/traversaro/gemma.cpp/build",
  "command": "/usr/bin/c++ -DHWY_STATIC_DEFINE -DTOOLCHAIN_MISS_ASM_HWCAP_H -I/home/traversaro/gemma.cpp/. -I/home/traversaro/gemma.cpp/build/_deps/sentencepiece-src -I/home/traversaro/gemma.cpp/build/_deps/highway-src -O3 -DNDEBUG -O2 -std=gnu++17 -o CMakeFiles/libgemma.dir/gemma.cc.o -c /home/traversaro/gemma.cpp/gemma.cc",
  "file": "/home/traversaro/gemma.cpp/gemma.cc"
},

If instead for some reason you want to use -O2, probably it is necessary to make sure that CMake do not pass -O3 in the first place, for example tweaking CMAKE_CXX_FLAGS_RELEASE or similar.