Addressing the challenge of high annotation costs in solving Math Word
Problems (MWPs) through full supervision with intermediate equations, recent
works have proposed weakly supervised task settings that rely solely on the
final answer as a supervised signal. Existing leading approaches typically
employ various search techniques to infer intermediate equations, but cannot
ensure their semantic consistency with natural language descriptions. The rise
of Large Language Models (LLMs) like ChatGPT has opened up new possibilities
for addressing MWPs directly. However, the computational demands of LLMs make
them less than ideal for use in settings where resources are tight. In light of
these challenges, we introduce an innovative two-stage framework that adeptly
transfers mathematical Expertise from large to tiny language models. In
\emph{Distillation Stage}, we propose a series of extraction processes that
satisfy the properties of MWPs to distill mathematical knowledge from LLMs to
construct problem-equation pairs required for supervised training. In
\emph{Refinement Stage}, Due to Knowledge distilling method cannot guarantee
the full utilization of all data, we further utilize the unsuccessfully
searched data effectively by Knowledge Refine method. Finally, We train a small
model using distilled data generated through two-stage methods. As our method
fully leverages the semantic understanding capabilities during the searching
'problem-equation' pair, it demonstrates significantly improved performance on
the Math23K and Weak12K datasets compared to existing small model methods,
while maintaining a much lower computational cost than ChatGPT.

通过创新的两阶段框架，从大型到小型语言模型中转移数学专业知识，通过弱监督任务设置解决数学文字问题中标注成本高的挑战，并在降低计算成本的同时，在 Math23K 和 Weak12K 数据集上表现出显著改进的性能。