Repository navigation
rand=-1 to use a random seed, not rand=0 as documented #386
Description
Hello,
I use this library to use llama-cpp in Python.
It's working very well but the generation is always the same (no seed in the script). I use this function
from llama_cpp import Llama
llama = Llama("./path/models/7B/ggml-model-7b.bin")
gen = llama.create_completion("Cite the best movie citation you know.",
max_tokens=50,
temperature=0.95,
top_p=0.95)I ran it several times and it's always the same text generated. Is there like HuggingFace a do_sample parameter to trigger ? I didn't find anything in the documentation or the issues of this github (but maybe I missed it)
Thank you, whoever you are who will help me :)
Reacted by Paul Mullins
Activity
gjmulder commentedon Jun 16, 2023
Try setting
seed=0when you init the model:llama_cpp.llama.Llama.init
rmngllnn commentedon Jun 16, 2023
Same thing happens. Got the same generation with three runs.
gjmulder commentedon Jun 16, 2023
That's strange. Can you try -1?
It looks like seed is passed to
libllama.so:rmngllnn commentedon Jun 16, 2023
It's working by passing -1 to seed
According to documentation , default is to 1337 and 0 value for random seed.
-1 does allow to have different generation for each run.
Thank you.
gjmulder commentedon Jun 16, 2023
Bug in the docs then 😄
rmngllnn commentedon Jun 16, 2023
Apparently ... which is unfortunate ...
Thank you for your (fast) response and help :)
[-]Always the same generation[/-][+]rand=-1 to use a random seed, not rand=0 as documented[/+]batrasrishti commentedon Oct 9, 2023
How to get same generated answer every single time?
Add missing header for memcpy (abetlen#386)
Update docs. Closes abetlen#386
gerrynjenny commentedon Jul 25, 2024
The above pointers were not permalinks so the urls are now 404. So I'm sharing this for others looking for the seed code:
llama-cpp-python/llama_cpp/llama.py
Line 82 in a14b49d
llama-cpp-python/llama_cpp/llama.py
Line 163 in a14b49d
dentropy commentedon Nov 6, 2024
Seems like this was the magic variable we need to set, setting seed and temerature by themselves does not work
sswam commentedon May 26, 2025
Setting seed=-1 still does not give a random seed. The LLM behaves identically at each start up. I'm working around this by setting seed=random.randint(0, 2**32).