So, I did get some minor FA speed improvements for TG, but I don't see what else one could do, so I'll merge it.

Here is a performance comparison between baseline (Q8_0 K-cache, no FA, no MLA), MLA (Q8_0 K-cache) and FA (Q8_0 K and V cache) for DeepSeek-Lite running on a Ryzen-7950X CPU. Both graphs show the MLA and FA performance ratio to baseline.

First graph is prompt processing speed. We see FA giving a ~40% performance boost at 16k tokens compared to baseline. MLA is 2X slower than baseline and 2.8X slower than FA at 16k tokens.

The second graph is token generation speed (TG-64) after a prompt of a given length (i.e., TG speed as a function of the number of tokens in the KV cache). We do get some performance gains for very long prompts from FA (~10% at 16k tokens), but by far not as much as from MLA. MLA is 1.57X faster than baseline and 1.43X faster than FA at 16k tokens.

Recently I read somewhere that for the "common enterprise workflow" (whatever that means) the number of generated tokens is typically only about 10% of the prompt tokens. I don't know if that is true, but for the sake of argument, let's assume for a moment that it is. In that case the best way to measure overall model performance is to use llama-bench -pg Npp,Ntg, where Ntg=0.1*Npp is the number of generated tokens and Npp is the number of prompt tokens. The following graph shows PG performance as a function of prompt length. The black symbols are mainline llama.cpp build b9ab0a4d (4687) (most current version as of today), the red symbols are for baseline ik_llama.cpp (no FA, no MLA), the green symbols are for MLA, and the blue symbols are for FA from this PR. The model is DeepSeek-Lite quantized with IQ4_XS. All use Q8_0 for K cache, FA uses Q8_0 also for V cache. All runs are on a Ryzen-7950X CPU. If we buy the claim that Ntg ~ 0.1*Npp is the "typical enterprise workflow", then there is no benefit from MLA over baseline, while FA is ~26% better for long prompts. Mainline llama.cpp is, as usual, slower. 1.45X for short prompts, increasing to 1.7X slower for prompts with 16k tokens.