In this paper, we introduce Target-Aware Weighted Training (TAWT), a weighted training algorithm for cross-task learning based on minimizing a representation-based task distance between the source and target tasks. We show that TAWT is easy to implement, is computationally efficient, requires little hyperparameter tuning, and enjoys non-asymptotic learning-theoretic guarantees. The effectiveness of TAWT is corroborated through extensive experiments with BERT on four sequence tagging tasks in natural language processing (NLP), including part-of-speech (PoS) tagging, chunking, predicate detection, and named entity recognition (NER). As a byproduct, the proposed representation-based task distance allows one to reason in a theoretically principled way about several critical aspects of cross-task learning, such as the choice of the source data and the impact of fine-tuning

本文引入了面向目标的加权训练(TAWT)，一个基于最小化源任务和目标任务之间的表示距离的交叉任务学习的加权训练算法。通过使用自然语言处理（NLP）中的四个序列标记任务来进行广泛的实验，表明TAWT易于实现，具有计算效率，需要很少的超参数调整，并享有非渐近学习理论保证。同时提出的基于表示的任务距离能够以理论上的方式去考虑交叉任务学习的几个关键方面，例如源数据的选择和微调的影响。

跨任务学习的加权训练