While Large Language Models (LLMs) excel in several natural language
processing tasks, their size and inaccessibility present challenges for
extensive practical application. Previous studies acquire specialized skills
through distillation on LLMs, which result in trading generic abilities, called
model specialization. As for reasoning ability, chain-of-thought was
synthesized to subsequent distillation. However, due to hallucination,
synthetic chain-of-thought from LLMs contains faulty reasoning. These incorrect
reasoning steps damage the reasoning capability. To tackle above issues, we
propose Program-aided Distillation (PaD), which distills LLMs to obtain
specialized small models in reasoning tasks. In PaD, we strengthen specialized
models with program-aided reasoning, and help them overcome faulty reasoning
steps with automated error checking. Experimental results demonstrate that, on
the GSM8K benchmark, a 0.06B model using PaD can not only outperform certain
LLMs (e.g., LLaMA), but also achieves a 10% improvement over baselines with a
significantly smaller scale of parameters and data. Data pruning analysis
reveals that PaD possesses higher training efficiency.

提出了 Program-aided Distillation（PaD）算法，该算法使用程序辅助推理强化可特化的模型，并通过自动化错误检查帮助它们克服错误的推理步骤，实验证明 PaD 具有比 LLMs 更高的训练效率，并能显著提高推理能力。