Commits

Commit history

1. Commits on Jul 18, 2024

   • fix: Change repeat_penalty to 1.0 to match llama.cpp defaults (#1590)

     Show description for 0700476
     

     ddh0

     and

     abetlen

     authored
     0700476View commit details

     Copy full SHA for 0700476

     View code at this pointBrowse repository at this point
2. Commits on Jul 17, 2024

   • feat: Update llama.cpp

     abetlen

     committed
     7613d23View commit details

     Copy full SHA for 7613d23

     View code at this pointBrowse repository at this point
3. Commits on Jul 9, 2024

   • fix(minor): Minor ruff fixes

     abetlen

     committed
     08f2bb3View commit details

     Copy full SHA for 08f2bb3

     View code at this pointBrowse repository at this point

   • fix(misc): Format

     abetlen

     committed
     c1ae815View commit details

     Copy full SHA for c1ae815

     View code at this pointBrowse repository at this point
4. Commits on Jun 19, 2024

   • fix: Make destructor to automatically call .close() method on Llama class.

     abetlen

     committed
     4c1d74cView commit details

     Copy full SHA for 4c1d74c

     View code at this pointBrowse repository at this point
5. Commits on Jun 13, 2024

   • feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

     Show description for 320a5d7
     

     jkawamoto

     and

     abetlen

     authored
     320a5d7View commit details

     Copy full SHA for 320a5d7

     View code at this pointBrowse repository at this point

   • feat: Support SPM infill (#1492)

     Show description for dbcf64c
     

     CISC

     and

     abetlen

     authored
     dbcf64cView commit details

     Copy full SHA for dbcf64c

     View code at this pointBrowse repository at this point
6. Commits on Jun 4, 2024

   • feat: adding `rpc_servers` parameter to `Llama` class (#1477)

     Show description for d634efc
     

     chraac

     and

     abetlen

     authored
     d634efcView commit details

     Copy full SHA for d634efc

     View code at this pointBrowse repository at this point

   • fix: fix logprobs when BOS is not present (#1471)

     Show description for 6e0642c
     a-ghorbani

     authored
     6e0642cView commit details

     Copy full SHA for 6e0642c

     View code at this pointBrowse repository at this point

   • fix: Avoid duplicate special tokens in chat formats (#1439)

     Show description for 027f7bc
     

     CISC

     and

     abetlen

     authored
     027f7bcView commit details

     Copy full SHA for 027f7bc

     View code at this pointBrowse repository at this point
7. Commits on May 29, 2024

   • fix: fix string value kv_overrides. Closes #1487

     abetlen

     committed
     df45a4bView commit details

     Copy full SHA for df45a4b

     View code at this pointBrowse repository at this point
8. Commits on May 24, 2024

   • feat: Improve Llama.eval performance by avoiding list conversion (#1476)

     Show description for 5cae104
     

     thoughtp0lice

     and

     abetlen

     authored
     5cae104View commit details

     Copy full SHA for 5cae104

     View code at this pointBrowse repository at this point
9. Commits on May 16, 2024

   • fix: segfault for models without eos / bos tokens. Closes #1463

     abetlen

     committed
     d99a6baView commit details

     Copy full SHA for d99a6ba

     View code at this pointBrowse repository at this point
10. Commits on May 14, 2024

    • feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)

      Show description for 5212fb0
      twaka

      authored
      5212fb0View commit details

      Copy full SHA for 5212fb0

      View code at this pointBrowse repository at this point

    • misc: Remove unnecessary metadata lookups (#1448)

      Show description for 389e09c
      CISC

      authored
      389e09cView commit details

      Copy full SHA for 389e09c

      View code at this pointBrowse repository at this point
11. Commits on May 9, 2024

    • feat: Support multiple chat templates - step 1 (#1396)

      Show description for 5ab40e6
      

      CISC

      and

      abetlen

      authored
      5ab40e6View commit details

      Copy full SHA for 5ab40e6

      View code at this pointBrowse repository at this point
12. Commits on May 8, 2024

    • feat: fill-in-middle support (#1386)

      Show description for 4a7122d
      

      CISC

      and

      abetlen

      authored
      4a7122dView commit details

      Copy full SHA for 4a7122d

      View code at this pointBrowse repository at this point

    • fix: chat_format log where auto-detected format prints `None` (#1434)

      balvisio

      authored
      a50d24eView commit details

      Copy full SHA for a50d24e

      View code at this pointBrowse repository at this point
13. Commits on May 3, 2024

    • fix: Use memmove to copy str_value kv_override. Closes #1417

      abetlen

      committed
      9f7a855View commit details

      Copy full SHA for 9f7a855

      View code at this pointBrowse repository at this point
14. Commits on Apr 30, 2024

    • fix: wrong parameter for flash attention in pickle __getstate__

      abetlen

      committed
      29b6e9aView commit details

      Copy full SHA for 29b6e9a

      View code at this pointBrowse repository at this point

    • feat: Add option to enable `flash_attn` to Lllama params and ModelSettings

      abetlen

      committed
      22d77eeView commit details

      Copy full SHA for 22d77ee

      View code at this pointBrowse repository at this point
15. Commits on Apr 28, 2024

    • feat: Add support for str type kv_overrides

      abetlen

      committed
      a411612View commit details

      Copy full SHA for a411612

      View code at this pointBrowse repository at this point
16. Commits on Apr 26, 2024

    • feat: Allow for possibly non-pooled embeddings (#1380)

      Show description for f6ed21f
      

      iamlemec

      and

      abetlen

      authored
      f6ed21fView commit details

      Copy full SHA for f6ed21f

      View code at this pointBrowse repository at this point
17. Commits on Apr 22, 2024

    • feat: Use new llama_token_is_eog in create_completions

      abetlen

      committed
      d40a250View commit details

      Copy full SHA for d40a250

      View code at this pointBrowse repository at this point
18. Commits on Apr 20, 2024

    • feat: Add stopping_criteria to ChatFormatter, allow stopping on arbitrary token ids, fixes llama3 instruct

      abetlen

      committed
      cc81afeView commit details

      Copy full SHA for cc81afe

      View code at this pointBrowse repository at this point
19. Commits on Apr 17, 2024

    • feat: Make saved state more compact on-disk (#1296)

      Show description for 4924455
      tc-wolf

      authored
      4924455View commit details

      Copy full SHA for 4924455

      View code at this pointBrowse repository at this point

    • feat: Use all available CPUs for batch processing (#1345)

      

      ddh0

      authored and

      abetlen

      committed
      c96b2daView commit details

      Copy full SHA for c96b2da

      View code at this pointBrowse repository at this point
20. Commits on Apr 10, 2024

    • fix: pass correct type to chat handlers for chat completion logprobs

      abetlen

      committed
      bb65b4dView commit details

      Copy full SHA for bb65b4d

      View code at this pointBrowse repository at this point
21. Commits on Apr 3, 2024

    • fix: segfault when logits_all=False. Closes #1319

      abetlen

      committed
      8649d76View commit details

      Copy full SHA for 8649d76

      View code at this pointBrowse repository at this point
22. Commits on Apr 1, 2024

    • feat: add support for KV cache quantization options (#1307)

      Show description for f165048
      

      Limour-dev

      and

      abetlen

      authored
      f165048View commit details

      Copy full SHA for f165048

      View code at this pointBrowse repository at this point
23. Commits on Mar 31, 2024

    • feat: Add logprobs support to chat completions (#1311)

      Show description for aa9f1ae
      

      windspirit95

      and

      abetlen

      authored
      aa9f1aeView commit details

      Copy full SHA for aa9f1ae

      View code at this pointBrowse repository at this point
24. Commits on Mar 14, 2024

    • fix: set default pooling type to unspecified

      abetlen

      committed
      4084aabView commit details

      Copy full SHA for 4084aab

      View code at this pointBrowse repository at this point

    • fix: Set default pooling_type to mean, check for null pointer.

      abetlen

      committed
      d318cc8View commit details

      Copy full SHA for d318cc8

      View code at this pointBrowse repository at this point
25. Commits on Mar 9, 2024

    • feat: Switch embed to llama_get_embeddings_seq (#1263)

      Show description for 2811014
      

      iamlemec

      and

      abetlen

      authored
      2811014View commit details

      Copy full SHA for 2811014

      View code at this pointBrowse repository at this point
26. Commits on Mar 6, 2024

    • Update llama.cpp

      abetlen

      committed
      93dc56aView commit details

      Copy full SHA for 93dc56a

      View code at this pointBrowse repository at this point