This issue is caused by hwy::ConvertScalarTo called in EmbeddingScaling. The comment on EmbeddingScaling said that it's for rounding the float to bf16 to match Gemma's Embedder. It calls this version of the template function:

template <typename TTo, typename TFrom, HWY_IF_BF16(TFrom),
          HWY_IF_NOT_SPECIAL_FLOAT(TTo)>
HWY_API HWY_BF16_CONSTEXPR TTo ConvertScalarTo(TFrom in) {
  return static_cast<TTo>(F32FromBF16(in));
}

When HWY_COMPILER_GCC_ACTUAL < 1300 and there is no built-in function __builtin_bit_cast, It will not be constexpr. However, constexpr attribute of EmbeddingScaling only follows Sqrt, which may cause compilation errors. So I open this PR to fix it.