Skip to content

Tags: quantumaikr/quant.cpp

Tags

v0.13.0

Toggle v0.13.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
chore: bump version to v0.13.0 + release notes (#81)

v0.13.0 highlights:
- Phi-3 / Phi-3.5 architecture fully supported (fused QKV/FFN, LongRoPE)
- Phi-3.5-mini Q8_0 as default model (2x faster than Q4_K_M on NEON)
- quant-server-unified (quant.h-based, no sync divergence)
- ChatML template marker filter (BPE-split stop detection)
- 16 chat-cache bugs eliminated across 2 audit passes
- ChatContextOverflow exception in Python

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

v0.12.1

Toggle v0.12.1's commit message
v0.12.1: quantcpp client (SSE streaming) + serve UX

- New: quantcpp client PROMPT — SSE streaming HTTP client
- Improved: quantcpp serve startup output (curl examples, SDK snippet)
- Server already supported SSE streaming (stream:true) — now discoverable

v0.12.0

Toggle v0.12.0's commit message
v0.12.0: ollama-parity CLI (quantcpp pull/list/run/serve)

PyPI CLI now supports ollama-style commands:
  pip install quantcpp
  quantcpp pull llama3.2:1b
  quantcpp list
  quantcpp run llama3.2:1b
  quantcpp serve llama3.2:1b --port 8080

Short aliases: smollm2:135m, qwen3.5:0.8b, llama3.2:1b
Auto-pull on run/serve. Backwards-compatible with v0.11.x default chat mode.

v0.8.0

Toggle v0.8.0's commit message
v0.8.0: AVX2 turbo_kv port + Metal diag + KL tool

Bundles four work items completed in one session:
  1. AVX2 port of turbo_kv_4b/5b/5b_fast/3b attention (commit 2dcbde4)
  2. Metal flush diagnostic counter for Issue #16 (commit 34f5ef4)
  3. KL divergence two-pass tool for llama.cpp PR validation (fd4148b)
  4. v0.9.0 vdotq experiment: explored, measured (-1.5% PPL), reverted

CHANGELOG documents each item with measurements where available.
The llama.cpp PR draft is updated to mark KL divergence as DONE
(was the main remaining blocker).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

v0.7.2

Toggle v0.7.2's commit message
v0.7.2 — turbo_kv_5b_fast: near-lossless at parity speed (1-byte layout)

v0.7.1

Toggle v0.7.1's commit message
v0.7.1 — Round 11: NEON tbl applied to 3b/5b (partial parity)

v0.7.0

Toggle v0.7.0's commit message
v0.7.0 — turbo_kv_4b PARITY with fp32 KV (Round 10 NEON tbl)

v0.6.5

Toggle v0.6.5's commit message
v0.6.5 — re-baseline benchmarks without Metal (Metal is slower at bat…

…ch-1)

v0.6.4

Toggle v0.6.4's commit message
v0.6.4 — honest validation pass; corrects v0.6.3 speed claims

v0.6.3

Toggle v0.6.3's commit message
v0.6.3 — turbo_kv beats fp32 KV speed (Karpathy round 5+6)