Commits

Commit history

1. Commits on Jun 19, 2024

   • fix: Make destructor to automatically call .close() method on Llama class.

     abetlen

     committed
     4c1d74cView commit details

     Copy full SHA for 4c1d74c

     View code at this pointBrowse repository at this point
2. Commits on Jun 13, 2024

   • feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)

     Show description for 320a5d7
     

     jkawamoto

     and

     abetlen

     authored
     320a5d7View commit details

     Copy full SHA for 320a5d7

     View code at this pointBrowse repository at this point

   • feat: Support SPM infill (#1492)

     Show description for dbcf64c
     

     CISC

     and

     abetlen

     authored
     dbcf64cView commit details

     Copy full SHA for dbcf64c

     View code at this pointBrowse repository at this point
3. Commits on Jun 4, 2024

   • feat: adding `rpc_servers` parameter to `Llama` class (#1477)

     Show description for d634efc
     

     chraac

     and

     abetlen

     authored
     d634efcView commit details

     Copy full SHA for d634efc

     View code at this pointBrowse repository at this point

   • fix: fix logprobs when BOS is not present (#1471)

     Show description for 6e0642c
     a-ghorbani

     authored
     6e0642cView commit details

     Copy full SHA for 6e0642c

     View code at this pointBrowse repository at this point

   • fix: Avoid duplicate special tokens in chat formats (#1439)

     Show description for 027f7bc
     

     CISC

     and

     abetlen

     authored
     027f7bcView commit details

     Copy full SHA for 027f7bc

     View code at this pointBrowse repository at this point
4. Commits on May 29, 2024

   • fix: fix string value kv_overrides. Closes #1487

     abetlen

     committed
     df45a4bView commit details

     Copy full SHA for df45a4b

     View code at this pointBrowse repository at this point
5. Commits on May 24, 2024

   • feat: Improve Llama.eval performance by avoiding list conversion (#1476)

     Show description for 5cae104
     

     thoughtp0lice

     and

     abetlen

     authored
     5cae104View commit details

     Copy full SHA for 5cae104

     View code at this pointBrowse repository at this point
6. Commits on May 16, 2024

   • fix: segfault for models without eos / bos tokens. Closes #1463

     abetlen

     committed
     d99a6baView commit details

     Copy full SHA for d99a6ba

     View code at this pointBrowse repository at this point
7. Commits on May 14, 2024

   • feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)

     Show description for 5212fb0
     twaka

     authored
     5212fb0View commit details

     Copy full SHA for 5212fb0

     View code at this pointBrowse repository at this point

   • misc: Remove unnecessary metadata lookups (#1448)

     Show description for 389e09c
     CISC

     authored
     389e09cView commit details

     Copy full SHA for 389e09c

     View code at this pointBrowse repository at this point
8. Commits on May 9, 2024

   • feat: Support multiple chat templates - step 1 (#1396)

     Show description for 5ab40e6
     

     CISC

     and

     abetlen

     authored
     5ab40e6View commit details

     Copy full SHA for 5ab40e6

     View code at this pointBrowse repository at this point
9. Commits on May 8, 2024

   • feat: fill-in-middle support (#1386)

     Show description for 4a7122d
     

     CISC

     and

     abetlen

     authored
     4a7122dView commit details

     Copy full SHA for 4a7122d

     View code at this pointBrowse repository at this point

   • fix: chat_format log where auto-detected format prints `None` (#1434)

     balvisio

     authored
     a50d24eView commit details

     Copy full SHA for a50d24e

     View code at this pointBrowse repository at this point
10. Commits on May 3, 2024

    • fix: Use memmove to copy str_value kv_override. Closes #1417

      abetlen

      committed
      9f7a855View commit details

      Copy full SHA for 9f7a855

      View code at this pointBrowse repository at this point
11. Commits on Apr 30, 2024

    • fix: wrong parameter for flash attention in pickle __getstate__

      abetlen

      committed
      29b6e9aView commit details

      Copy full SHA for 29b6e9a

      View code at this pointBrowse repository at this point

    • feat: Add option to enable `flash_attn` to Lllama params and ModelSettings

      abetlen

      committed
      22d77eeView commit details

      Copy full SHA for 22d77ee

      View code at this pointBrowse repository at this point
12. Commits on Apr 28, 2024

    • feat: Add support for str type kv_overrides

      abetlen

      committed
      a411612View commit details

      Copy full SHA for a411612

      View code at this pointBrowse repository at this point
13. Commits on Apr 26, 2024

    • feat: Allow for possibly non-pooled embeddings (#1380)

      Show description for f6ed21f
      

      iamlemec

      and

      abetlen

      authored
      f6ed21fView commit details

      Copy full SHA for f6ed21f

      View code at this pointBrowse repository at this point
14. Commits on Apr 22, 2024

    • feat: Use new llama_token_is_eog in create_completions

      abetlen

      committed
      d40a250View commit details

      Copy full SHA for d40a250

      View code at this pointBrowse repository at this point
15. Commits on Apr 20, 2024

    • feat: Add stopping_criteria to ChatFormatter, allow stopping on arbitrary token ids, fixes llama3 instruct

      abetlen

      committed
      cc81afeView commit details

      Copy full SHA for cc81afe

      View code at this pointBrowse repository at this point
16. Commits on Apr 17, 2024

    • feat: Make saved state more compact on-disk (#1296)

      Show description for 4924455
      tc-wolf

      authored
      4924455View commit details

      Copy full SHA for 4924455

      View code at this pointBrowse repository at this point

    • feat: Use all available CPUs for batch processing (#1345)

      

      ddh0

      authored and

      abetlen

      committed
      c96b2daView commit details

      Copy full SHA for c96b2da

      View code at this pointBrowse repository at this point
17. Commits on Apr 10, 2024

    • fix: pass correct type to chat handlers for chat completion logprobs

      abetlen

      committed
      bb65b4dView commit details

      Copy full SHA for bb65b4d

      View code at this pointBrowse repository at this point
18. Commits on Apr 3, 2024

    • fix: segfault when logits_all=False. Closes #1319

      abetlen

      committed
      8649d76View commit details

      Copy full SHA for 8649d76

      View code at this pointBrowse repository at this point
19. Commits on Apr 1, 2024

    • feat: add support for KV cache quantization options (#1307)

      Show description for f165048
      

      Limour-dev

      and

      abetlen

      authored
      f165048View commit details

      Copy full SHA for f165048

      View code at this pointBrowse repository at this point
20. Commits on Mar 31, 2024

    • feat: Add logprobs support to chat completions (#1311)

      Show description for aa9f1ae
      

      windspirit95

      and

      abetlen

      authored
      aa9f1aeView commit details

      Copy full SHA for aa9f1ae

      View code at this pointBrowse repository at this point
21. Commits on Mar 14, 2024

    • fix: set default pooling type to unspecified

      abetlen

      committed
      4084aabView commit details

      Copy full SHA for 4084aab

      View code at this pointBrowse repository at this point

    • fix: Set default pooling_type to mean, check for null pointer.

      abetlen

      committed
      d318cc8View commit details

      Copy full SHA for d318cc8

      View code at this pointBrowse repository at this point
22. Commits on Mar 9, 2024

    • feat: Switch embed to llama_get_embeddings_seq (#1263)

      Show description for 2811014
      

      iamlemec

      and

      abetlen

      authored
      2811014View commit details

      Copy full SHA for 2811014

      View code at this pointBrowse repository at this point
23. Commits on Mar 6, 2024

    • Update llama.cpp

      abetlen

      committed
      93dc56aView commit details

      Copy full SHA for 93dc56a

      View code at this pointBrowse repository at this point
24. Commits on Mar 1, 2024

    • docs: Add information re: auto chat formats. Closes #1236

      abetlen

      committed
      97aa3a1View commit details

      Copy full SHA for 97aa3a1

      View code at this pointBrowse repository at this point

    • feat: Update llama.cpp

      abetlen

      committed
      f062a7fView commit details

      Copy full SHA for f062a7f

      View code at this pointBrowse repository at this point
25. Commits on Feb 28, 2024

    • fix: eos/bos_token set correctly for Jinja2ChatFormatter and automatic chat formatter (#1230)

      Show description for c36ab15
      CISC

      authored
      c36ab15View commit details

      Copy full SHA for c36ab15

      View code at this pointBrowse repository at this point
26. Commits on Feb 25, 2024

    • feat: Update llama.cpp

      abetlen

      committed
      2292af5View commit details

      Copy full SHA for 2292af5

      View code at this pointBrowse repository at this point