Learning from multiple related tasks by knowledge sharing and transfer has
become increasingly relevant over the last two decades. In order to
successfully transfer information from one task to another, it is critical to
understand the similarities and differences between the domains. In this paper,
we introduce the notion of \emph{performance gap}, an intuitive and novel
measure of the distance between learning tasks. Unlike existing measures which
are used as tools to bound the difference of expected risks between tasks
(e.g., $\mathcal{H}$-divergence or discrepancy distance), we theoretically show
that the performance gap can be viewed as a data- and algorithm-dependent
regularizer, which controls the model complexity and leads to finer guarantees.
More importantly, it also provides new insights and motivates a novel principle
for designing strategies for knowledge sharing and transfer: gap minimization.
We instantiate this principle with two algorithms: 1. gapBoost, a novel and
principled boosting algorithm that explicitly minimizes the performance gap
between source and target domains for transfer learning; and 2. gapMTNN, a
representation learning algorithm that reformulates gap minimization as
semantic conditional matching for multitask learning. Our extensive evaluation
on both transfer learning and multitask learning benchmark data sets shows that
our methods outperform existing baselines.

本文提出了性能差距这一新概念，作为数据和算法依赖的正则化因子，以控制模型复杂度，并提出了性能差距最小化原则，进一步设计了知识共享和转移的策略，并应用于两种算法：gapBoost 和 gapMTNN，通过在转移学习和多任务学习基准数据集上的广泛评估表明，这些方法优于现有的基线模型。