Deep learning systems thrive on abundance of labeled training data but such data is not always available, calling for alternative methods of supervision. One such method is expectation regularization (XR) (Mann and McCallum, 2007), where models are trained based on expected label proportions. We propose a novel application of the XR framework for transfer learning between related tasks, where knowing the labels of task A provides an estimation of the label proportion of task B. We then use a model trained for A to label a large corpus, and use this corpus with an XR loss to train a model for task B. To make the XR framework applicable to large-scale deep-learning setups, we propose a stochastic batched approximation procedure. We demonstrate the approach on the task of Aspect-based Sentiment classification, where we effectively use a sentence-level sentiment predictor to train accurate aspect-based predictor. The method improves upon fully supervised neural system trained on aspect-level data, and is also cumulative with LM-based pretraining, as we demonstrate by improving a BERT-based Aspect-based Sentiment model.

本文提出了一种基于预期标签比例的 expectation regularization (XR) 方法在相关任务之间进行迁移学习的应用，该方法通过一个用于A任务的模型来标记大语料库并生成B任务的标签比例，使XR框架适用于大规模深度学习，通过随机批近似过程进行。在情感分类任务中，我们利用句子级情感预测器对准确的基于方面的预测器进行训练，该方法优于使用方面级数据训练的全监督神经系统，并且与基于语言模型的预训练累计，我们利用改进BERT基于方面的情感模型的结果来证明这一点。

使用期望标签比例在相关任务之间进行迁移学习