A critical component of a successful language generation pipeline is the
decoding algorithm. However, the general principles that should guide the
choice of a decoding algorithm remain unclear. Previous works only compare
decoding algorithms in narrow scenarios, and their findings do not generalize
across tasks. We argue that the misalignment between the model's likelihood and
the task-specific notion of utility is the key factor to understanding the
effectiveness of decoding algorithms. To structure the discussion, we introduce
a taxonomy of misalignment mitigation strategies (MMSs), providing a unifying
view of decoding as a tool for alignment. The MMS taxonomy groups decoding
algorithms based on their implicit assumptions about likelihood--utility
misalignment, yielding general statements about their applicability across
tasks. Specifically, by analyzing the correlation between the likelihood and
the utility of predictions across a diverse set of tasks, we provide empirical
evidence supporting the proposed taxonomy and a set of principles to structure
reasoning when choosing a decoding algorithm. Crucially, our analysis is the
first to relate likelihood-based decoding algorithms with algorithms that rely
on external information, such as value-guided methods and prompting, and covers
the most diverse set of tasks to date. Code, data, and models are available at
this https URL.

这篇论文提出了一种消减可能存在的语言生成模型中的似然度与任务特定效用误差的分类方法，并分析了该方法在不同任务中的可行性，并提供了一组决策根据，决定选择哪种解码算法。