Commits

Commit history

1. Commits on Mar 30, 2026

   • feat(server): add model-load chat_template_kwargs (#2168)

     abetlen

     authored
     7257ba9View commit details

     Copy full SHA for 7257ba9

     View code at this pointBrowse repository at this point
2. Commits on Mar 22, 2026

   • misc: Add Ruff formatting (#2148)

     Show description for a9b4a06
     abetlen

     authored
     a9b4a06View commit details

     Copy full SHA for a9b4a06

     View code at this pointBrowse repository at this point
3. Commits on Jul 3, 2025

   • feat: Add support for new mtmd api, add Qwen2.5-VL chat handler

     abetlen

     committed
     cd548bdView commit details

     Copy full SHA for cd548bd

     View code at this pointBrowse repository at this point
4. Commits on Sep 20, 2024

   • feat: Add option to configure n_ubatch

     abetlen

     committed
     6c44a3fView commit details

     Copy full SHA for 6c44a3f

     View code at this pointBrowse repository at this point
5. Commits on Aug 29, 2024

   • feat: Add server chat_format minicpm-v-2.6 for MiniCPMv26ChatHandler

     abetlen

     committed
     97d527eView commit details

     Copy full SHA for 97d527e

     View code at this pointBrowse repository at this point
6. Commits on Jul 17, 2024

   • fix(server): Use split_mode from model settings (#1594)

     Show description for 66d5cdd
     

     grider-withourai

     and

     abetlen

     authored
     66d5cddView commit details

     Copy full SHA for 66d5cdd

     View code at this pointBrowse repository at this point
7. Commits on Jun 13, 2024

   • feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

     Show description for 320a5d7
     

     jkawamoto

     and

     abetlen

     authored
     320a5d7View commit details

     Copy full SHA for 320a5d7

     View code at this pointBrowse repository at this point
8. Commits on Jun 4, 2024

   • feat: adding `rpc_servers` parameter to `Llama` class (#1477)

     Show description for d634efc
     

     chraac

     and

     abetlen

     authored
     d634efcView commit details

     Copy full SHA for d634efc

     View code at this pointBrowse repository at this point
9. Commits on May 29, 2024

   • fix: fix string value kv_overrides. Closes #1487

     abetlen

     committed
     df45a4bView commit details

     Copy full SHA for df45a4b

     View code at this pointBrowse repository at this point
10. Commits on May 3, 2024

    • fix(server): Propagate `flash_attn` to model load. (#1424)

      dthuerck

      authored
      2138561View commit details

      Copy full SHA for 2138561

      View code at this pointBrowse repository at this point
11. Commits on May 2, 2024

    • feat: Add llama-3-vision-alpha chat format

      abetlen

      committed
      31b1d95View commit details

      Copy full SHA for 31b1d95

      View code at this pointBrowse repository at this point
12. Commits on Apr 30, 2024

    • feat: Generic Chat Formats, Tool Calling, and Huggingface Pull Support for Multimodal Models (Obsidian, LLaVA1.6, Moondream) (#1147)

      Show description for fe2da09
      abetlen

      authored
      fe2da09View commit details

      Copy full SHA for fe2da09

      View code at this pointBrowse repository at this point
13. Commits on Apr 1, 2024

    • feat: add support for KV cache quantization options (#1307)

      Show description for f165048
      

      Limour-dev

      and

      abetlen

      authored
      f165048View commit details

      Copy full SHA for f165048

      View code at this pointBrowse repository at this point
14. Commits on Feb 28, 2024

    • misc: Format

      abetlen

      committed
      727d60cView commit details

      Copy full SHA for 727d60c

      View code at this pointBrowse repository at this point
15. Commits on Feb 26, 2024

    • feat(server): Add support for pulling models from Huggingface Hub (#1222)

      Show description for 4d574bd
      abetlen

      authored
      4d574bdView commit details

      Copy full SHA for 4d574bd

      View code at this pointBrowse repository at this point
16. Commits on Feb 8, 2024

    • fix: broken import

      abetlen

      committed
      85d3374View commit details

      Copy full SHA for 85d3374

      View code at this pointBrowse repository at this point

    • feat: Integrate functionary v1.4 and v2 models + add custom tokenizer support to Llama class (#1078)

      Show description for 9018270
      

      jeffrey-fong

      and

      abetlen

      authored
      9018270View commit details

      Copy full SHA for 9018270

      View code at this pointBrowse repository at this point
17. Commits on Jan 31, 2024

    • Add speculative decoding (#1120)

      Show description for fb762a6
      abetlen

      authored
      fb762a6View commit details

      Copy full SHA for fb762a6

      View code at this pointBrowse repository at this point
18. Commits on Jan 21, 2024

    • fix: pass chat handler not chat formatter for huggingface autotokenizer and tokenizer_config formats.

      abetlen

      committed
      24f3945View commit details

      Copy full SHA for 24f3945

      View code at this pointBrowse repository at this point
19. Commits on Jan 19, 2024

    • feat: Add ability to load chat format from huggingface autotokenizer or tokenizer_config.json files.

      abetlen

      committed
      b8fc1c7View commit details

      Copy full SHA for b8fc1c7

      View code at this pointBrowse repository at this point
20. Commits on Jan 15, 2024

    • Implement GGUF metadata KV overrides (#1011)

      Show description for 76aafa6
      

      phiharri

      and

      abetlen

      authored
      76aafa6View commit details

      Copy full SHA for 76aafa6

      View code at this pointBrowse repository at this point
21. Commits on Dec 22, 2023

    • [Feat] Multi model support (#931)

      Show description for 12b7f2f
      

      D4ve-R

      and

      abetlen

      authored
      12b7f2fView commit details

      Copy full SHA for 12b7f2f

      View code at this pointBrowse repository at this point
22. Loading