Deploying large language models (LLMs) poses challenges in terms of resource
limitations and inference efficiency. To address these challenges, recent
research has focused on using smaller task-specific language models, which are
enhanced by distilling the knowledge rationales generated by LLMs. However,
previous works mostly emphasize the effectiveness of positive knowledge, while
overlooking the knowledge noise and the exploration of negative knowledge. In
this paper, we first propose a general approach called quality-guided
contrastive rationale distillation for reasoning capacity learning, considering
contrastive learning perspectives. For the learning of positive knowledge, we
collect positive rationales through self-consistency to denoise the LLM
rationales generated by temperature sampling. For the negative knowledge
distillation, we generate negative rationales using temperature sampling for
the iteration-before smaller language models themselves. Finally, a contrastive
loss is designed to better distill the positive and negative rationales into
the smaller language model, where an online-update discriminator is used to
judge the qualities of rationales and assign weights for better optimizing the
training process. Through extensive experiments on multiple reasoning tasks, we
demonstrate that our method consistently outperforms the previous distillation
methods and produces higher-quality rationales.

通过对比性理由蒸馏，我们提出了一种质量引导的方法用于推理能力学习，能够从大型语言模型中提取正面和负面的知识理由，并通过在线更新的判别器优化训练过程，得到更高质量的可解释性底层语言模型。