Addressing the challenge of high annotation costs in solving Math Word
Problems (MWPs) through full supervision with intermediate equations, recent
works have proposed weakly supervised task settings that rely solely on the
final answer as a supervised signal. Existing leading approaches typically
employ various search techniques to infer intermediate equations, but cannot
ensure their semantic consistency with natural language descriptions. The rise
of Large Language Models (LLMs) like ChatGPT has opened up new possibilities
for addressing MWPs directly. However, the computational demands of LLMs make
them less than ideal for use in settings where resources are tight. In light of
these challenges, we introduce an innovative two-stage framework that adeptly
transfers mathematical Expertise from large to tiny language models. In
\emph{Distillation Stage}, we propose a series of extraction processes that
satisfy the properties of MWPs to distill mathematical knowledge from LLMs to
construct problem-equation pairs required for supervised training. In
\emph{Refinement Stage}, Due to Knowledge distilling method cannot guarantee
the full utilization of all data, we further utilize the unsuccessfully
searched data effectively by Knowledge Refine method. Finally, We train a small
model using distilled data generated through two-stage methods. As our method
fully leverages the semantic understanding capabilities during the searching
'problem-equation' pair, it demonstrates significantly improved performance on
the Math23K and Weak12K datasets compared to existing small model methods,
while maintaining a much lower computational cost than ChatGPT.

通过创新的两阶段框架，从大型到小型语言模型中转移数学专业知识，通过弱监督任务设置解决数学文字问题中标注成本高的挑战，并在降低计算成本的同时，在 Math23K 和 Weak12K 数据集上表现出显著改进的性能。

由大到小：利用弱监督提炼和提高数学词问题的数学专业知识

From Large to Tiny: Distilling and Refining Mathematical Expertise for  Math Word Problems with Weakly Supervision

The pretrain-finetune paradigm in modern computer vision facilitates the
success of self-supervised learning, which tends to achieve better
transferability than supervised learning. However, with the availability of
massive labeled data, a natural question emerges: how to train a better model
with both self and full supervision signals? In this paper, we propose
Omni-suPErvised Representation leArning with hierarchical supervisions (OPERA)
as a solution. We provide a unified perspective of supervisions from labeled
and unlabeled data and propose a unified framework of fully supervised and
self-supervised learning. We extract a set of hierarchical proxy
representations for each image and impose self and full supervisions on the
corresponding proxy representations. Extensive experiments on both
convolutional neural networks and vision transformers demonstrate the
superiority of OPERA in image classification, segmentation, and object
detection. Code is available at: this https URL.

该研究提出了一种名为 OPERA 的 Omni-suPErvised Representation leArning 方法，通过全面利用受监督和无监督的数据，将自监督学习和完全监督学习统一到一个框架中，以获得更好的模型性能。该方法通过使用层次化监督对每个图像提取一组代理表示，并在相应的代理表示上施加自监督和完全监督信号，展现了在图像分类、分割和目标检测上性能的稳定提升。

OPERA: 带有层级监督的全方位监督表示学习

OPERA: Omni-Supervised Representation Learning with Hierarchical Supervisions

Weakly-supervised object localization (WSOL) has gained popularity over the
last years for its promise to train localization models with only image-level
labels. Since the seminal WSOL work of class activation mapping (CAM), the
field has focused on how to expand the attention regions to cover objects more
broadly and localize them better. However, these strategies rely on full
localization supervision for validating hyperparameters and model selection,
which is in principle prohibited under the WSOL setup. In this paper, we argue
that WSOL task is ill-posed with only image-level labels, and propose a new
evaluation protocol where full supervision is limited to only a small held-out
set not overlapping with the test set. We observe that, under our protocol, the
five most recent WSOL methods have not made a major improvement over the CAM
baseline. Moreover, we report that existing WSOL methods have not reached the
few-shot learning baseline, where the full-supervision at validation time is
used for model training instead. Based on our findings, we discuss some future
directions for WSOL.

本文提出一个新的评估协议，将完全监督仅限于一个小的保留数据集，观察了最近五种 WSOL 方法与 CAM 基线的比较并发现没有显著的改进，并提出未来 WSOL 的研究方向建议。