Large Language Models (LLMs) generate longform text by successively sampling
the next token based on the probability distribution of the token vocabulary at
each decoding step. Current popular truncation sampling methods such as top-$p$
sampling, also known as nucleus sampling, often struggle to balance coherence
and creativity in generating text, particularly when using higher temperatures.
To address this issue, we propose min-$p$, a dynamic truncation sampling
method, that establishes a minimum base percentage threshold for tokens, which
the scales according to the probability of the top candidate token. Through
experiments on several benchmarks, such as GPQA, GSM8K and AlpacaEval Creative
Writing, we demonstrate that min-$p$ improves the coherence and quality of
generated text even at high temperatures, while also facilitating more creative
and diverse outputs compared to top-$p$ and other sampling methods. As of
writing, min-$p$ has been adopted by multiple open-source LLM implementations,
and have been independently assessed by members of the open-source LLM
community, further validating its practical utility and potential.

提出了一种动态截断采样方法 min-$p$，通过在生成文本时根据候选词的概率来建立最小基础百分比阈值来平衡自洽性和创造力，实验表明 min-$p$ 在高温度下提高了生成文本的一致性和质量，同时与 top-$p$ 和其他采样方法相比，还能产生更具创意和多样性的文本输出。