Eliciting "chain of thought" (CoT) rationales -- sequences of token that
convey a "reasoning" process -- has been shown to consistently improve LLM
performance on tasks like question answering. More recent efforts have shown
that such rationales can also be used for model distillation: Including CoT
sequences (elicited from a large "teacher" model) in addition to target labels
when fine-tuning a small student model yields (often substantial) improvements.
In this work we ask: Why and how does this additional training signal help in
model distillation? We perform ablations to interrogate this, and report some
potentially surprising results. Specifically: (1) Placing CoT sequences after
labels (rather than before) realizes consistently better downstream performance
-- this means that no student "reasoning" is necessary at test time to realize
gains. (2) When rationales are appended in this way, they need not be coherent
reasoning sequences to yield improvements; performance increases are robust to
permutations of CoT tokens, for example. In fact, (3) a small number of key
tokens are sufficient to achieve improvements equivalent to those observed when
full rationales are used in model distillation.

链式思维 (Chain of Thought) 的理由序列已被证明能在问题回答等任务中有效提高语言模型的性能。最近的研究表明，这种理由序列在模型蒸馏中也能起到作用。通过将这些序列（由大型 “教师” 模型生成）与目标标签一同用于微调小型 “学生” 模型，能显著提高性能。本研究旨在探讨为何以及如何这种额外的训练信号能在模型蒸馏中发挥作用，并报道了一些有趣的结果。