Repository navigation
Upgrade llama.cpp from b12283 to b12289 - #613
Conversation
Six upstream commits: OCP FP8 E4M3 and new flash-attention cases in the CUDA module, MMVF for BF16, upstream's tests, two WebUI components. No header this library includes moved; all 20 patches apply unchanged. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
The generator writes the per-layer k_scale / v_scale tensors; the model answers with f8_e4m3 on both sides and on one, with flash attention on and off, the same from a restored slot, and under F16 like the model without scales. A scale of zero fails the load with the 8-bit float cache only. 732 tests, 776 cases. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
|
Generated by Claude Code |
|



Summary
03d6c824e1ccbdb8e45b82299b1965cf94c5c056), six upstream commits after b12283. They touch onlyggml/src/ggml-cuda/*,ggml/CMakeLists.txt,ggml/cmake/common.cmake, upstream'stests/and two WebUI files: an FP8 E4M3 KV cache on CUDA, andggml-cudanow linksCUDA::cublasLt. No file a local patch touches changed; all 20 patches apply unchanged, and the WebUI lockfile is unchanged.blk.N.attn_k.k_scale,attn_v.v_scale) an 8-bit float cache divides K and V by, and twoCApiFp8Cachetests use them: a model with scales answers withf8_e4m3on both sides and on one, with flash attention on and off, the same from a saved and restored slot, and under an F16 cache exactly like the model without scales; a scale of zero fails the load with the 8-bit float cache only. No test reached that path before.docs/history/llama-cpp-breaking-changes.md,LlamaCppVersionand the natives NOTICE follow the pin.Test plan
NativeLibraryLoadSmokeTest,LlamaCppVersionTest;check-natives.py,check-patches.py, the buildcheck unit testsNot measured: the CUDA side of the bump (the FP8 cache kernels, the
cublasLtlink) -- no GPU here or on the runners; the two CUDA module jobs build it. A CUDA consumer now needscublasLt64_13.dll/libcublasLt.so.13from the toolkit, which the README already lists for Windows.Related issues / PRs
Follows #612.
Checklist
CONTRIBUTING.mdandCODE_OF_CONDUCT.mdGenerated by Claude Code