Large language models (LLMs) exhibit impressive emergent abilities in natural
language processing, but their democratization is hindered due to huge
computation requirements and closed-source nature. Recent research on advancing
open-source smaller LMs by distilling knowledge from black-box LLMs has
obtained promising results in the instruction-following ability. However, the
reasoning ability which is more challenging to foster, is relatively rarely
explored. In this paper, we propose a tailored learning approach to distill
such reasoning ability to smaller LMs to facilitate the democratization of the
exclusive reasoning ability. In contrast to merely employing LLM as a data
annotator, we exploit the potential of LLM as a reasoning teacher by building
an interactive multi-round learning paradigm. This paradigm enables the student
to expose its deficiencies to the black-box teacher who then can provide
customized training data in return. Further, to exploit the reasoning potential
of the smaller LM, we propose self-reflection learning to motivate the student
to learn from self-made mistakes. The learning from self-reflection and LLM are
all tailored to the student's learning status, thanks to the seamless
integration with the multi-round learning paradigm. Comprehensive experiments
and analysis on mathematical and commonsense reasoning tasks demonstrate the
effectiveness of our method. The code will be available at
this https URL

通过多轮互动学习范式和自我反思学习，我们提出了一种定制化学习方法，从而将推理能力传授给较小的语言模型，以促进推理能力的开放和普及。