Existing Large Language Models (LLMs) generate text through unidirectional
autoregressive decoding methods to respond to various user queries. These
methods tend to consider token selection in a simple sequential manner, making
it easy to fall into suboptimal options when encountering uncertain tokens,
referred to as chaotic points in our work. Many chaotic points exist in texts
generated by LLMs, and they often significantly affect the quality of
subsequently generated tokens, which can interfere with LLMs' generation. This
paper proposes Self-Evaluation Decoding, SED, a decoding method for enhancing
model generation. Analogous to the human decision-making process, SED
integrates speculation and evaluation steps into the decoding process, allowing
LLMs to make more careful decisions and thus optimize token selection at
chaotic points. Experimental results across various tasks using different LLMs
demonstrate SED's effectiveness.

现有的大语言模型通过单向自回归解码方法生成文本，但这种方法在处理不确定 token 时容易遇到次优选择，而该研究提出了一种名为 SED 的自我评估解码方法以优化模型生成。实验结果表明 SED 在各种任务和不同模型上具有良好的效果。