a/Next-Dining-The-Video-Game
1
1
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

Don't set seed since llama.cpp behaves nondeterministically for prompts > ≈228 tokens for some reason

Browse source
This commit is contained in:
Anthony Wang 2024-02-05 21:56:01 -05:00
parent 1ca92f001a
commit e82765c654
Signed by: a
SSH key fingerprint: SHA256:B5ADfMCqd2M7d/jtXDoihAV/yfXOAbWWri9+GdCN4hQ
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
billions.py
View file

@ -30,7 +30,7 @@ for i in options:
c = inp
prompt += i[0] + ": " + c + "\n"
prompt += "\nSure! Here's the extremely long and detailed 2468-word book you requested:\n\n"
r = requests.post("http://boss-baby:8080/completion", json={"prompt": prompt, "temperature": 1.0, "repeat_last_n": 1024, "seed": 1, "stream": True}, headers={"Authorization": "Bearer 12345"}, stream=True)
r = requests.post("http://boss-baby:8080/completion", json={"prompt": prompt, "temperature": 1.0, "repeat_last_n": 1024, "stream": True}, headers={"Authorization": "Bearer 12345"}, stream=True)
r.encoding = 'utf-8'
for chunk in r.iter_lines(decode_unicode=True):
if chunk != "":