In this paper, we investigate the phenomena of "selection biases" in Large
Language Models (LLMs), focusing on problems where models are tasked with
choosing the optimal option from an ordered sequence. We delve into biases
related to option order and token usage, which significantly impact LLMs'
decision-making processes. We also quantify the impact of these biases through
an extensive empirical analysis across multiple models and tasks. Furthermore,
we propose mitigation strategies to enhance model performance. Our key
contributions are threefold: 1) Precisely quantifying the influence of option
order and token on LLMs, 2) Developing strategies to mitigate the impact of
token and order sensitivity to enhance robustness, and 3) Offering a detailed
analysis of sensitivity across models and tasks, which informs the creation of
more stable and reliable LLM applications for selection problems.

我们在这篇论文中研究了大型语言模型（LLMs）中的 “选择偏差” 现象，专注于模型在从有序序列中选择最佳选项的问题。我们深入探讨了与选项顺序和标记使用相关的偏差，这些偏差显著影响了 LLMs 的决策过程。通过多个模型和任务的广泛实证分析，我们还对这些偏差的影响进行了量化。此外，我们提出了缓解策略以增强模型性能。我们的主要贡献有三个方面：1）精确量化了选项顺序和标记对 LLMs 的影响；2）开发了缓解标记和顺序敏感性以增强鲁棒性的策略；3）详细分析了不同模型和任务的敏感性，为选择问题的更稳定可靠的 LLMs 应用程序的创建提供了指导。