Commits

Commit history

1. Commits on Jun 1, 2026

   • fix: correct typo in comments and settings description (#2121)

     Show description for 33bf9d2
     thecaptain789

     authored
     33bf9d2View commit details

     Copy full SHA for 33bf9d2

     View code at this pointBrowse repository at this point
2. Commits on Mar 30, 2026

   • feat(server): add model-load chat_template_kwargs (#2168)

     abetlen

     authored
     7257ba9View commit details

     Copy full SHA for 7257ba9

     View code at this pointBrowse repository at this point
3. Commits on Sep 20, 2024

   • feat: Add option to configure n_ubatch

     abetlen

     committed
     6c44a3fView commit details

     Copy full SHA for 6c44a3f

     View code at this pointBrowse repository at this point
4. Commits on Jul 9, 2024

   • fix(misc): Format

     abetlen

     committed
     c1ae815View commit details

     Copy full SHA for c1ae815

     View code at this pointBrowse repository at this point
5. Commits on Jul 2, 2024

   • fix(server): Update `embeddings=False` by default. Embeddings should be enabled by default for embedding models.

     abetlen

     committed
     bf5e0bbView commit details

     Copy full SHA for bf5e0bb

     View code at this pointBrowse repository at this point
6. Commits on Jun 4, 2024

   • feat: adding `rpc_servers` parameter to `Llama` class (#1477)

     Show description for d634efc
     

     chraac

     and

     abetlen

     authored
     d634efcView commit details

     Copy full SHA for d634efc

     View code at this pointBrowse repository at this point
7. Commits on May 5, 2024

   • feat(server): Add support for setting root_path. Closes #1420

     abetlen

     committed
     0318702View commit details

     Copy full SHA for 0318702

     View code at this pointBrowse repository at this point
8. Commits on Apr 30, 2024

   • feat: Add option to enable `flash_attn` to Lllama params and ModelSettings

     abetlen

     committed
     22d77eeView commit details

     Copy full SHA for 22d77ee

     View code at this pointBrowse repository at this point
9. Commits on Apr 26, 2024

   • fix: pydantic deprecation warning

     abetlen

     committed
     fcfea66View commit details

     Copy full SHA for fcfea66

     View code at this pointBrowse repository at this point
10. Commits on Apr 23, 2024

    • feat(server): Provide ability to dynamically allocate all threads if desired using `-1` (#1364)

      sean-bailey

      authored
      53ebcc8View commit details

      Copy full SHA for 53ebcc8

      View code at this pointBrowse repository at this point
11. Commits on Apr 17, 2024

    • feat: add `disable_ping_events` flag (#1257)

      Show description for b73c73c
      khimaros

      authored
      b73c73cView commit details

      Copy full SHA for b73c73c

      View code at this pointBrowse repository at this point

    • feat: Use all available CPUs for batch processing (#1345)

      

      ddh0

      authored and

      abetlen

      committed
      c96b2daView commit details

      Copy full SHA for c96b2da

      View code at this pointBrowse repository at this point
12. Commits on Apr 1, 2024

    • feat: add support for KV cache quantization options (#1307)

      Show description for f165048
      

      Limour-dev

      and

      abetlen

      authored
      f165048View commit details

      Copy full SHA for f165048

      View code at this pointBrowse repository at this point
13. Commits on Feb 28, 2024

    • misc: Format

      abetlen

      committed
      727d60cView commit details

      Copy full SHA for 727d60c

      View code at this pointBrowse repository at this point

    • feat: Update llama.cpp

      abetlen

      committed
      0d37ce5View commit details

      Copy full SHA for 0d37ce5

      View code at this pointBrowse repository at this point
14. Commits on Feb 26, 2024

    • feat(server): Add support for pulling models from Huggingface Hub (#1222)

      Show description for 4d574bd
      abetlen

      authored
      4d574bdView commit details

      Copy full SHA for 4d574bd

      View code at this pointBrowse repository at this point

    • fix: remove prematurely commited change

      abetlen

      committed
      dcf38f6View commit details

      Copy full SHA for dcf38f6

      View code at this pointBrowse repository at this point
15. Commits on Feb 25, 2024

    • feat: Update llama.cpp

      abetlen

      committed
      2292af5View commit details

      Copy full SHA for 2292af5

      View code at this pointBrowse repository at this point
16. Commits on Feb 17, 2024

    • feat: Update llama.cpp

      abetlen

      committed
      fdce078View commit details

      Copy full SHA for fdce078

      View code at this pointBrowse repository at this point
17. Commits on Jan 31, 2024

    • Add speculative decoding (#1120)

      Show description for fb762a6
      abetlen

      authored
      fb762a6View commit details

      Copy full SHA for fb762a6

      View code at this pointBrowse repository at this point
18. Commits on Jan 29, 2024

    • Automatically set chat format from gguf (#1110)

      Show description for da003d8
      abetlen

      authored
      da003d8View commit details

      Copy full SHA for da003d8

      View code at this pointBrowse repository at this point
19. Commits on Jan 19, 2024

    • feat: Add ability to load chat format from huggingface autotokenizer or tokenizer_config.json files.

      abetlen

      committed
      b8fc1c7View commit details

      Copy full SHA for b8fc1c7

      View code at this pointBrowse repository at this point
20. Commits on Jan 18, 2024

    • Offload KQV by default

      abetlen

      committed
      48c3b77View commit details

      Copy full SHA for 48c3b77

      View code at this pointBrowse repository at this point
21. Commits on Jan 16, 2024

    • fix(cli): allow passing n_ctx=0 to openAI API server args to use model n_ctx_train field per #1015 (#1093)

      K-Mistele

      authored
      9c36688View commit details

      Copy full SHA for 9c36688

      View code at this pointBrowse repository at this point
22. Commits on Jan 15, 2024

    • Add split_mode option. Closes #1085

      abetlen

      committed
      84615adView commit details

      Copy full SHA for 84615ad

      View code at this pointBrowse repository at this point

    • Implement GGUF metadata KV overrides (#1011)

      Show description for 76aafa6
      

      phiharri

      and

      abetlen

      authored
      76aafa6View commit details

      Copy full SHA for 76aafa6

      View code at this pointBrowse repository at this point
23. Commits on Dec 22, 2023

    • docs: add server config docs

      abetlen

      committed
      522aecbView commit details

      Copy full SHA for 522aecb

      View code at this pointBrowse repository at this point

    • [Feat] Multi model support (#931)

      Show description for 12b7f2f
      

      D4ve-R

      and

      abetlen

      authored
      12b7f2fView commit details

      Copy full SHA for 12b7f2f

      View code at this pointBrowse repository at this point
24. Loading