Despite the success of distillation in large language models (LLMs), most prior work applies identical loss functions to both teacher- and student-generated data. These strategies overlook the synergy between loss formulations and data types, leading to a suboptimal performance boost in student models. To address this, we propose DistiLLM-2, a contrastive approach that simultaneously increases the likelihood of teacher responses and decreases that of student responses by harnessing this synergy. Our extensive experiments show that DistiLLM-2 not only builds high-performing student models across a wide range of tasks, including instruction-following and code generation, but also supports diverse applications, such as preference alignment and vision-language extensions. These findings highlight the potential of a contrastive approach to enhance the efficacy of LLM distillation by effectively aligning teacher and student models across varied data types.

本研究解决了在大语言模型蒸馏中不同数据类型与损失函数协同作用的不足，提出了一种对比方法DistiLLM-2。该方法通过增加教师模型响应的可能性并降低学生模型响应的可能性，显著提升了学生模型的表现，使其在多种任务中表现出色，并且能够支持如偏好对齐和视觉语言扩展等不同应用。

对比方法提升大语言模型的蒸馏效果：DistiLLM-2