Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. It leverages or reuses rich labeled data from the different but related source domain(s) and unlabeled data from the target domain. To this end, previous work focuses on either extracting domain-invariant features or task-agnostic features, ignoring domain-aware features that may be present in the target domain and could be useful for the downstream task. In this paper, we propose a two-stage framework for cross-domain text classification. In the first stage, we finetune the model with mask language modeling (MLM) and labeled data from the source domain. In the second stage, we further fine-tune the model with self-supervised distillation (SSD) and unlabeled data from the target domain. We evaluate its performance on a public cross-domain text classification benchmark and the experiment results show that our method achieves new state-of-the-art results for both single-source domain adaptations (94.17% $\uparrow$1.03%) and multi-source domain adaptations (95.09% $\uparrow$1.34%).

本文提出了一种二阶段跨域文本分类的框架，使用掩蔽语言模型和标记数据从源域进行模型微调，然后使用自监督蒸馏和目标域的无标注数据进一步微调模型，实验结果表明我们的方法在单源域（94.17％ ↑1.03％）和多源域（95.09％ ↑1.34％）适应都达到了最新的最先进水平。

自监督蒸馏的两阶段跨领域文本分类框架