Commits

Breadcrumbs

History for

onmain

User selector

All users

All time

Commit History

Commits on Feb 9, 2026

• Refactor Llama class to use new LlamaSampler chain API from _internals

  Show description for 1e6094a
  JamePeng

  committed
  1e6094a

  Copy full SHA for 1e6094a

Commits on Jan 29, 2026

• Sync llama : disable Direct IO by default

  JamePeng

  committed
  5bdff0b

  Copy full SHA for 5bdff0b

Commits on Jan 8, 2026

• Update llama.cpp API 20260108

  JamePeng

  committed
  4d0fe24

  Copy full SHA for 4d0fe24

Commits on Jan 1, 2026

• Update llama_context_params and fix the `embeddings` typo

  JamePeng

  committed
  b6bf564

  Copy full SHA for b6bf564

Commits on Nov 2, 2025

• Implement LlamaTrieCache into llama_cache.py: Optimize LlamaCache lookup from O(N) to O(K) using a Trie

  JamePeng

  committed
  2419dc2

  Copy full SHA for 2419dc2

Commits on Aug 31, 2025

• Sync llama: use FA + max. GPU layers by default

  JamePeng

  committed
  2be720d

  Copy full SHA for 2be720d

Commits on Aug 26, 2025

• Sync llama : remove KV cache defragmentation logic

  JamePeng

  committed
  ae46275

  Copy full SHA for ae46275

Commits on Aug 10, 2025

• Added gemma3 setting.chat_format into server/model.py

  JamePeng

  committed
  9645ded

  Copy full SHA for 9645ded

Commits on Jul 17, 2025

• fix memory_seq_rm crash bug

  JamePeng

  committed
  8d981f0

  Copy full SHA for 8d981f0

Commits on Jul 16, 2025

• Update llama.cpp API 20250717

  JamePeng

  committed
  bc283d3

  Copy full SHA for bc283d3

Commits on Jul 13, 2025

• more logits_all params restored

  JamePeng

  committed
  cdc7348

  Copy full SHA for cdc7348

Commits on Jul 6, 2025

• Sync abetlen/llama-cpp-python 0.3.11 code

  JamePeng

  committed
  0701c73

  Copy full SHA for 0701c73

Commits on May 20, 2025

• Sync kv-cache : add SWA support

  JamePeng

  committed
  177445c

  Copy full SHA for 177445c

Commits on May 13, 2025

• Update llama.cpp API code 20250513

  JamePeng

  committed
  6b3e11c

  Copy full SHA for 6b3e11c

Commits on May 9, 2025

• Sync context : remove logits_all flag and update API

  JamePeng

  committed
  1a1d920

  Copy full SHA for 1a1d920

Commits on Jan 8, 2025

• fix: streaming resource lock (#1879)

  Show description for e8f14ce
  

  gjpower

  and

  abetlen

  authored
  e8f14ce

  Copy full SHA for e8f14ce

Commits on Dec 9, 2024

• fix: add missing await statements for async exit_stack handling (#1858)

  gjpower

  authored
  afedfc8

  Copy full SHA for afedfc8

Commits on Dec 6, 2024

• fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context (#1798)

  Show description for 9bd0c95
  

  gjpower

  and

  abetlen

  authored
  9bd0c95

  Copy full SHA for 9bd0c95

• fix: added missing exit_stack.close() to /v1/chat/completions (#1796)

  Show description for 073b7e4
  Ian321

  authored
  073b7e4

  Copy full SHA for 073b7e4

Commits on Sep 20, 2024

• feat: Add option to configure n_ubatch

  abetlen

  committed
  6c44a3f

  Copy full SHA for 6c44a3f

Commits on Aug 29, 2024

• feat: Add server chat_format minicpm-v-2.6 for MiniCPMv26ChatHandler

  abetlen

  committed
  97d527e

  Copy full SHA for 97d527e

Commits on Jul 17, 2024

• fix(server): Use split_mode from model settings (#1594)

  Show description for 66d5cdd
  

  grider-withourai

  and

  abetlen

  authored
  66d5cdd

  Copy full SHA for 66d5cdd

Commits on Jul 9, 2024

• fix(misc): Format

  abetlen

  committed
  c1ae815

  Copy full SHA for c1ae815

Commits on Jul 2, 2024

• fix(misc): Fix type errors

  abetlen

  committed
  387d01d

  Copy full SHA for 387d01d

• fix(server): Fix bug in FastAPI streaming response where dependency was released before request completes causing SEGFAULT

  abetlen

  committed
  296304b

  Copy full SHA for 296304b

• fix(server): Update `embeddings=False` by default. Embeddings should be enabled by default for embedding models.

  abetlen

  committed
  bf5e0bb

  Copy full SHA for bf5e0bb

Commits on Jun 13, 2024

• feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

  Show description for 320a5d7
  

  jkawamoto

  and

  abetlen

  authored
  320a5d7

  Copy full SHA for 320a5d7

Commits on Jun 4, 2024

• feat: adding `rpc_servers` parameter to `Llama` class (#1477)

  Show description for d634efc
  

  chraac

  and

  abetlen

  authored
  d634efc

  Copy full SHA for d634efc

Commits on May 29, 2024

• fix: fix string value kv_overrides. Closes #1487

  abetlen

  committed
  df45a4b

  Copy full SHA for df45a4b

Commits on May 14, 2024

• feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)

  Show description for 5212fb0
  twaka

  authored
  5212fb0

  Copy full SHA for 5212fb0

Commits on May 5, 2024

• feat(server): Add support for setting root_path. Closes #1420

  abetlen

  committed
  0318702

  Copy full SHA for 0318702

Commits on May 3, 2024

• Merge branch 'main' of github.com:abetlen/llama_cpp_python into main

  abetlen

  committed
  f9b7221

  Copy full SHA for f9b7221

• feat(server): Remove temperature bounds checks for server. Closes #1384

  abetlen

  committed
  0a454be

  Copy full SHA for 0a454be

• fix(server): Propagate `flash_attn` to model load. (#1424)

  dthuerck

  authored
  2138561

  Copy full SHA for 2138561

Commits on May 2, 2024

• feat: Add llama-3-vision-alpha chat format

  abetlen

  committed
  31b1d95

  Copy full SHA for 31b1d95