Skip to content

Upgrade llama.cpp to b12309; Host/Origin guard in OpenAiCompatServer; quantizer records the matrix by file name - #614

Merged
bernardladenthin merged 4 commits into
mainfrom
claude/wonderful-tesla-1fkezz
Nov 11, 2026
Merged

bernardladenthin merged 4 commits into
mainfrom
claude/wonderful-tesla-1fkezz

Conversation

@bernardladenthin

Copy link
Copy Markdown
Owner

Summary

  • llama.cpp b12289 -> b12292 -> b12309 (2cbe0ec429b849a88664e0f62388fc92a5abd699; upstream has not tagged b12309 yet, so it is pinned by commit, and the configure checks that the commit's count is the number). 20 upstream commits in all. No patched file is in the range: all 20 patches apply unchanged, and no header this library includes moved. What arrives with it: flash attention on the CPU sums an F16 V cache in F32, a tts template can normalize its text itself, Jinja chains comparisons and unpacks arguments, --cors-origins on the embedded server refuses a foreign Host / Origin with 403 (default * unchanged), cpp-httplib 0.66.0, and GPU-module changes (CUDA graphs, ROCm BLAS workspace, Vulkan batched copy and dequantized KV cache with coopmat2, Metal SOLVE_TRI).
  • OpenAiCompatServer refuses requests from somewhere else once corsAllowOrigin is not *. Until now the value only set Access-Control-Allow-Origin: a browser hid the response from a foreign page and the request still ran. The new pure OriginGuard applies the rules of llama.cpp's own server (upstream #32098): a foreign Origin or Sec-Fetch-Site, and on a loopback connection a Host that names neither loopback nor the host of an allowed origin (DNS rebinding), are answered 403 permission_error before any route runs. The value may be localhost or a comma-separated list; new CLI option --cors-origins.
  • The quantizer records the importance matrix by file name. quantize() wrote the path it was given into quantize.imatrix.file, so a published GGUF named the directories of the machine it was made on. Upstream fixed the same in its command-line tool (#32000), which this library does not use.
  • Tests: NativeServerCors (3, the real llama_server() with and without --cors-origins), CApiTemplate (a chat template with a chained comparison and unpacked arguments), the quantizer test now asserts the recorded value, OriginGuardTest (10), OpenAiCompatServerOriginTest (5, over a socket), one CLI test.

Behaviour change: a server configured with a specific corsAllowOrigin answered every request before; a browser request from another origin is now 403, a list or localhost is answered with the one origin that asked, and Vary: Origin is set. The default * and every client that is no browser are unchanged.

Test plan

  • C++ suite locally (Linux x86-64, CPU, static build against b12309 with the patches applied): 780/780 ctest cases, 736 test definitions
  • Java: the server tests (OriginGuardTest, OpenAiCompatServer*Test, OpenAiServerCliTest), WireNameRegistryTest; spotbugs:check, spotless
  • check-natives.py, check-patches.py, the buildcheck unit tests
  • CI is green on this branch
  • Docs / CHANGELOG updated

Not covered: OriginGuard has no browser test (the rules are tested over a socket); the tts template normalizer (no generated speech model) and the F32 accumulation (needs a context long enough to overflow F16) are not testable here; the CUDA, ROCm, Vulkan and Metal changes are not measured on a GPU -- the module jobs build them.

Related issues / PRs

Follows #613.

Checklist

  • I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
  • My commits follow Conventional Commits
  • No security-sensitive changes that need private disclosure (the guard is a hardening, default behaviour unchanged)

Generated by Claude Code

Three upstream commits, in the Vulkan and WebGPU backends and upstream's
test-backend-ops. No patched file is in the range; all 20 patches apply
unchanged, and no source of this library changes.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
Seventeen upstream commits, pinned by commit (upstream has not tagged the
number yet). No patched file is in the range; all 20 patches apply
unchanged, and no header this library includes moved.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
…gured origin

With corsAllowOrigin other than "*" the server only set the
Access-Control-Allow-Origin header: a browser hid the response from a
foreign page, and the request ran. OriginGuard applies the rules of
llama.cpp's --cors-origins (upstream #32098): a foreign Origin or
Sec-Fetch-Site, and on a loopback connection a Host that names neither
loopback nor the host of an allowed origin (DNS rebinding), are answered
403 before any route runs. The value may be "localhost" or a
comma-separated list; --cors-origins sets it on the command line. The
default "*" is unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
quantize() wrote the path it was given into quantize.imatrix.file, so a
published GGUF named the directories of the machine it was made on.
Upstream fixed the same in its tool (#32000), which this library does
not use; both keys hold the file name now.

New in the C++ suite: NativeServerCors (--cors-origins on the embedded
server, upstream #32098) and a chat template with a chained comparison
and unpacked arguments (#32072). 736 tests, 780 cases.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S8cAMZXymmPba3YZr2YCWp
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Explain Change →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copy link
Copy Markdown
Owner Author

claude-review is red and is not this PR's: the action ends with Claude execution failed: result is_error:true before it reads the diff, as on main, #612 and #613. It is the review workflow's own Claude call, not code in this change; no fix for it exists in the repository, and I cannot re-run it from here. The same holds for submit-maven and FOSSA's "License Compliance" should they report red here as they do on main. Everything else on this PR I watch and fix.


Generated by Claude Code

@bernardladenthin
bernardladenthin merged commit 6ce3389 into main Nov 11, 2026
13 of 18 checks passed
@bernardladenthin
bernardladenthin deleted the claude/wonderful-tesla-1fkezz branch November 11, 2026 21:26
@sonarqubecloud

Copy link
Copy Markdown

This branch was successfully deployed

No deployments
startgate — c8bbd1a0 Deployed Nov 11, 2026 by bernardladenthin via Start gate (abort window) #1498
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants