For open-ended language generation tasks such as storytelling and dialogue,
choosing the right decoding algorithm is critical to controlling the tradeoff
between generation quality and diversity. However, there presently exists no
consensus on which decoding procedure is best or even the criteria by which to
compare them. We address these issues by casting decoding as a multi-objective
optimization problem aiming to simultaneously maximize both response quality
and diversity. Our framework enables us to perform the first large-scale
evaluation of decoding methods along the entire quality-diversity spectrum. We
find that when diversity is a priority, all methods perform similarly, but when
quality is viewed as more important, the recently proposed nucleus sampling
(Holtzman et al. 2019) outperforms all other evaluated decoding algorithms. Our
experiments also confirm the existence of the `likelihood trap', the
counter-intuitive observation that high likelihood sequences are often
surprisingly low quality. We leverage our findings to create and evaluate an
algorithm called \emph{selective sampling} which tractably approximates
globally-normalized temperature sampling.

将解码算法视为多目标优化问题，旨在同时最大化回应质量和多样性。当多样性是一个优先考虑的因素时，所有方法表现相似，但当质量被视为更重要时，近期提出的核采样 (nucleus sampling) 优于所有其他评估的解码算法。在这一发现的基础上，我们设计并评估了一种名为 “选择性抽样” 的算法，该算法可以近似全局归一化温度抽样。