What should Top-k be set to?
#23
by
daydreamwarrior
- opened
I was looking at the discussion here huggingface.co/Nanbeige/Nanbeige4.1-3B/discussions/2, and the conclusion seems to be either 0 or 50.
The discussion mentions that 50 is likely used because the official example utilizes the transformers defaults. However, since that example didn't set Temperature or Top-p either, it doesn't seem very convincing/conclusive.
So, what is the recommended value for Top-k? Thanks.
After testing it for a while on same promts, i assume the best settings are --temp 0.6 --top-p 0.95 --top-k 40 --min-p 0.01
Just by looking at output quality...