Update low-level examples to use new sampling API #141

Activity

1. mikeggh commented on May 3, 2023

   

   mikeggh

   on May 3, 2023

   More actions

   I'd like to know an example for using:
   llama_load_session_file(ctx, session_file, embd, out_len, (llama_cpp.c_int * out_len)(*embd_inp_out))

   and save of course... if you get a chance :> i'll try to brute force it later though..
2. SagsMug commented on May 3, 2023

   

   SagsMug

   on May 3, 2023

   Contributor

   More actions

   Im gonna work on updating the chat example, it might take a moment given the number of changes 😉

   👍React with 👍1Reacted by Andrei
3. abetlen commented on May 3, 2023

   

   abetlen

   on May 3, 2023

   OwnerAuthor

   More actions

   Thanks @SagsMug yeah I was a little overwhelmed with the new sampling API but overall it seems like the right direction to have more control over sampling. Check out https://github.com/abetlen/llama-cpp-python/blob/main/llama_cpp/llama.py#L255 might help

   👍React with 👍1Reacted by mikeggh
4. abetlen

   closed this as completedin #153on May 8, 2023