We investigate multi-task learning approaches that use a shared feature
representation for all tasks. To better understand the transfer of task
information, we study an architecture with a shared module for all tasks and a
separate output module for each task. We study the theory of this setting on
linear and ReLU-activated models. Our key observation is that whether or not
tasks' data are well-aligned can significantly affect the performance of
multi-task learning. We show that misalignment between task data can cause
negative transfer (or hurt performance) and provide sufficient conditions for
positive transfer. Inspired by the theoretical insights, we show that aligning
tasks' embedding layers leads to performance gains for multi-task training and
transfer learning on the GLUE benchmark and sentiment analysis tasks; for
example, we obtain a 2.35% GLUE score average improvement on 5 GLUE tasks over
BERT-LARGE using our alignment method. We also design an SVD-based task
reweighting scheme and show that it improves the robustness of multi-task
training on a multi-label image dataset.

本文研究了多任务学习的方法，使用共享特征表示所有任务，并研究了线性和 ReLU 激活模型的理论。通过实验证明了任务数据的对齐有助于多任务训练和转移学习，特别地，我们的对齐方法比 BERT-LARGE 在 5 个 GLUE 任务上平均提高了 2.35％的得分，并设计了基于 SVD 的任务重新加权方案来提高多任务训练在多标签图像数据集上的鲁棒性。

多任务学习中信息传输的理解与改善

Understanding and Improving Information Transfer in Multi-Task Learning

Recently, considerable effort has been devoted to deep domain adaptation in
computer vision and machine learning communities. However, most of existing
work only concentrates on learning shared feature representation by minimizing
the distribution discrepancy across different domains. Due to the fact that all
the domain alignment approaches can only reduce, but not remove the domain
shift. Target domain samples distributed near the edge of the clusters, or far
from their corresponding class centers are easily to be misclassified by the
hyperplane learned from the source domain. To alleviate this issue, we propose
to joint domain alignment and discriminative feature learning, which could
benefit both domain alignment and final classification. Specifically, an
instance-based discriminative feature learning method and a center-based
discriminative feature learning method are proposed, both of which guarantee
the domain invariant features with better intra-class compactness and
inter-class separability. Extensive experiments show that learning the
discriminative features in the shared feature space can significantly boost the
performance of deep domain adaptation methods.

通过联合域对齐和辨别特征学习的方式，可以带来对域对齐和最终任务分类的双重优化。对于视觉和机器学习领域，大部分现有工作只集中在通过最小化不同域之间的分布差异来学习共享的特征表示。然而，本文指出，所有域对齐方法只能减少，而不是消除域移位的情况。因此，我们提出了一种实例和中心的辨别特征学习方法，它们都可以在共享的特征空间中学习到具有更好的类内紧密性和类间可分性的域不变特征。实验表明，在共享的特征空间中学习辨别特征可以显著提高深度域自适应方法的性能。