Decoding methods play an indispensable role in converting language models
from next-token predictors into practical task solvers. Prior research on
decoding methods, primarily focusing on task-specific models, may not extend to
the current era of general-purpose large language models (LLMs). Moreover, the
recent influx of decoding strategies has further complicated this landscape.
This paper provides a comprehensive and multifaceted analysis of various
decoding methods within the context of LLMs, evaluating their performance,
robustness to hyperparameter changes, and decoding speeds across a wide range
of tasks, models, and deployment environments. Our findings reveal that
decoding method performance is notably task-dependent and influenced by factors
such as alignment, model size, and quantization. Intriguingly, sensitivity
analysis exposes that certain methods achieve superior performance at the cost
of extensive hyperparameter tuning, highlighting the trade-off between
attaining optimal results and the practicality of implementation in varying
contexts.

该研究通过全面而多方位的分析，评估了大型语言模型在各种任务、模型和部署环境下的解码方法的性能表现、对超参数变化的鲁棒性和解码速度，发现解码方法的性能与任务相关，并受到对齐、模型规模和量化等因素的影响。有趣的敏感性分析揭示了某些方法在广泛超参数调节的代价下实现了更优的性能，凸显了在不同环境中实现最优结果与实施实用性之间的权衡。