Commits

Commit history

1. Commits on Jan 8, 2026

   • Update llama.cpp API 20260108

     JamePeng

     committed
     4d0fe24View commit details

     Copy full SHA for 4d0fe24

     View code at this pointBrowse repository at this point
2. Commits on Jan 1, 2026

   • Update llama_context_params and fix the `embeddings` typo

     JamePeng

     committed
     b6bf564View commit details

     Copy full SHA for b6bf564

     View code at this pointBrowse repository at this point
3. Commits on Nov 2, 2025

   • Implement LlamaTrieCache into llama_cache.py: Optimize LlamaCache lookup from O(N) to O(K) using a Trie

     JamePeng

     committed
     2419dc2View commit details

     Copy full SHA for 2419dc2

     View code at this pointBrowse repository at this point
4. Commits on Aug 31, 2025

   • Sync llama: use FA + max. GPU layers by default

     JamePeng

     committed
     2be720dView commit details

     Copy full SHA for 2be720d

     View code at this pointBrowse repository at this point
5. Commits on Aug 26, 2025

   • Sync llama : remove KV cache defragmentation logic

     JamePeng

     committed
     ae46275View commit details

     Copy full SHA for ae46275

     View code at this pointBrowse repository at this point
6. Commits on Aug 10, 2025

   • Added gemma3 setting.chat_format into server/model.py

     JamePeng

     committed
     9645dedView commit details

     Copy full SHA for 9645ded

     View code at this pointBrowse repository at this point
7. Commits on Jul 17, 2025

   • fix memory_seq_rm crash bug

     JamePeng

     committed
     8d981f0View commit details

     Copy full SHA for 8d981f0

     View code at this pointBrowse repository at this point
8. Commits on Jul 16, 2025

   • Update llama.cpp API 20250717

     JamePeng

     committed
     bc283d3View commit details

     Copy full SHA for bc283d3

     View code at this pointBrowse repository at this point
9. Commits on Jul 13, 2025

   • more logits_all params restored

     JamePeng

     committed
     cdc7348View commit details

     Copy full SHA for cdc7348

     View code at this pointBrowse repository at this point
10. Commits on Jul 6, 2025

    • Sync abetlen/llama-cpp-python 0.3.11 code

      JamePeng

      committed
      0701c73View commit details

      Copy full SHA for 0701c73

      View code at this pointBrowse repository at this point
11. Commits on May 20, 2025

    • Sync kv-cache : add SWA support

      JamePeng

      committed
      177445cView commit details

      Copy full SHA for 177445c

      View code at this pointBrowse repository at this point
12. Commits on May 13, 2025

    • Update llama.cpp API code 20250513

      JamePeng

      committed
      6b3e11cView commit details

      Copy full SHA for 6b3e11c

      View code at this pointBrowse repository at this point
13. Commits on May 9, 2025

    • Sync context : remove logits_all flag and update API

      JamePeng

      committed
      1a1d920View commit details

      Copy full SHA for 1a1d920

      View code at this pointBrowse repository at this point
14. Commits on Jan 8, 2025

    • fix: streaming resource lock (#1879)

      Show description for e8f14ce
      

      gjpower

      and

      abetlen

      authored
      e8f14ceView commit details

      Copy full SHA for e8f14ce

      View code at this pointBrowse repository at this point
15. Commits on Dec 9, 2024

    • fix: add missing await statements for async exit_stack handling (#1858)

      gjpower

      authored
      afedfc8View commit details

      Copy full SHA for afedfc8

      View code at this pointBrowse repository at this point
16. Commits on Dec 6, 2024

    • fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context (#1798)

      Show description for 9bd0c95
      

      gjpower

      and

      abetlen

      authored
      9bd0c95View commit details

      Copy full SHA for 9bd0c95

      View code at this pointBrowse repository at this point

    • fix: added missing exit_stack.close() to /v1/chat/completions (#1796)

      Show description for 073b7e4
      Ian321

      authored
      073b7e4View commit details

      Copy full SHA for 073b7e4

      View code at this pointBrowse repository at this point
17. Commits on Sep 20, 2024

    • feat: Add option to configure n_ubatch

      abetlen

      committed
      6c44a3fView commit details

      Copy full SHA for 6c44a3f

      View code at this pointBrowse repository at this point
18. Commits on Aug 29, 2024

    • feat: Add server chat_format minicpm-v-2.6 for MiniCPMv26ChatHandler

      abetlen

      committed
      97d527eView commit details

      Copy full SHA for 97d527e

      View code at this pointBrowse repository at this point
19. Commits on Jul 17, 2024

    • fix(server): Use split_mode from model settings (#1594)

      Show description for 66d5cdd
      

      grider-withourai

      and

      abetlen

      authored
      66d5cddView commit details

      Copy full SHA for 66d5cdd

      View code at this pointBrowse repository at this point
20. Commits on Jul 9, 2024

    • fix(misc): Format

      abetlen

      committed
      c1ae815View commit details

      Copy full SHA for c1ae815

      View code at this pointBrowse repository at this point
21. Commits on Jul 2, 2024

    • fix(misc): Fix type errors

      abetlen

      committed
      387d01dView commit details

      Copy full SHA for 387d01d

      View code at this pointBrowse repository at this point

    • fix(server): Fix bug in FastAPI streaming response where dependency was released before request completes causing SEGFAULT

      abetlen

      committed
      296304bView commit details

      Copy full SHA for 296304b

      View code at this pointBrowse repository at this point

    • fix(server): Update `embeddings=False` by default. Embeddings should be enabled by default for embedding models.

      abetlen

      committed
      bf5e0bbView commit details

      Copy full SHA for bf5e0bb

      View code at this pointBrowse repository at this point
22. Commits on Jun 13, 2024

    • feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

      Show description for 320a5d7
      

      jkawamoto

      and

      abetlen

      authored
      320a5d7View commit details

      Copy full SHA for 320a5d7

      View code at this pointBrowse repository at this point
23. Commits on Jun 4, 2024

    • feat: adding `rpc_servers` parameter to `Llama` class (#1477)

      Show description for d634efc
      

      chraac

      and

      abetlen

      authored
      d634efcView commit details

      Copy full SHA for d634efc

      View code at this pointBrowse repository at this point
24. Commits on May 29, 2024

    • fix: fix string value kv_overrides. Closes #1487

      abetlen

      committed
      df45a4bView commit details

      Copy full SHA for df45a4b

      View code at this pointBrowse repository at this point
25. Commits on May 14, 2024

    • feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)

      Show description for 5212fb0
      twaka

      authored
      5212fb0View commit details

      Copy full SHA for 5212fb0

      View code at this pointBrowse repository at this point
26. Commits on May 5, 2024

    • feat(server): Add support for setting root_path. Closes #1420

      abetlen

      committed
      0318702View commit details

      Copy full SHA for 0318702

      View code at this pointBrowse repository at this point
27. Commits on May 3, 2024

    • Merge branch 'main' of github.com:abetlen/llama_cpp_python into main

      abetlen

      committed
      f9b7221View commit details

      Copy full SHA for f9b7221

      View code at this pointBrowse repository at this point

    • feat(server): Remove temperature bounds checks for server. Closes #1384

      abetlen

      committed
      0a454beView commit details

      Copy full SHA for 0a454be

      View code at this pointBrowse repository at this point

    • fix(server): Propagate `flash_attn` to model load. (#1424)

      dthuerck

      authored
      2138561View commit details

      Copy full SHA for 2138561

      View code at this pointBrowse repository at this point
28. Commits on May 2, 2024

    • feat: Add llama-3-vision-alpha chat format

      abetlen

      committed
      31b1d95View commit details

      Copy full SHA for 31b1d95

      View code at this pointBrowse repository at this point
29. Commits on Apr 30, 2024

    • feat: Add option to enable `flash_attn` to Lllama params and ModelSettings

      abetlen

      committed
      22d77eeView commit details

      Copy full SHA for 22d77ee

      View code at this pointBrowse repository at this point

    • feat: Generic Chat Formats, Tool Calling, and Huggingface Pull Support for Multimodal Models (Obsidian, LLaVA1.6, Moondream) (#1147)

      Show description for fe2da09
      abetlen

      authored
      fe2da09View commit details

      Copy full SHA for fe2da09

      View code at this pointBrowse repository at this point