Multiple-Choice Questions (MCQs) constitute a critical area of research in
the study of Large Language Models (LLMs). Previous works have investigated the
selection bias problem in MCQs within few-shot scenarios, in which the LLM's
performance may be influenced by the presentation of answer choices, leaving
the selection bias during Supervised Fine-Tuning (SFT) unexplored. In this
paper, we reveal that selection bias persists in the SFT phase , primarily due
to the LLM's inadequate Multiple Choice Symbol Binding (MCSB) ability. This
limitation implies that the model struggles to associate the answer options
with their corresponding symbols (e.g., A/B/C/D) effectively. To enhance the
model's MCSB capability, we first incorporate option contents into the loss
function and subsequently adjust the weights of the option symbols and
contents, guiding the model to understand the option content of the current
symbol. Based on this, we introduce an efficient SFT algorithm for MCQs, termed
Point-wise Intelligent Feedback (PIF). PIF constructs negative instances by
randomly combining the incorrect option contents with all candidate symbols,
and proposes a point-wise loss to provide feedback on these negative samples
into LLMs. Our experimental results demonstrate that PIF significantly reduces
the model's selection bias by improving its MCSB capability. Remarkably, PIF
exhibits a substantial enhancement in the accuracy for MCQs.

本文研究了大型语言模型中多项选择题的选择偏差问题，提出了一种基于点对点反馈的有效 SFT 算法（PIF），通过提高模型对选项内容的理解能力，显著减少了选择偏差，提高了多项选择题的准确性。