Transfer learning has emerged as a powerful technique for improving the performance of machine learning models on new domains where labeled training data may be scarce. In this approach a model trained for a source task, where plenty of labeled training data is available, is used as a starting point for training a model on a related target task with only few labeled training data. Despite recent empirical success of transfer learning approaches, the benefits and fundamental limits of transfer learning are poorly understood. In this paper we develop a statistical minimax framework to characterize the fundamental limits of transfer learning in the context of regression with linear and one-hidden layer neural network models. Specifically, we derive a lower-bound for the target generalization error achievable by any algorithm as a function of the number of labeled source and target data as well as appropriate notions of similarity between the source and target tasks. Our lower bound provides new insights into the benefits and limitations of transfer learning. We further corroborate our theoretical finding with various experiments.

本研究开发了一个统计极小化框架以表征在回归中通过线性和单隐藏层神经网络模型进行的迁移学习的基本限制，并提取出源数据和目标数据的标记数以及适当的相似性概念作为算法所能实现的目标泛化误差的下限。我们的下限提供了迁移学习的益处和限制的新见解，并通过各种实验验证了我们的理论发现。

线性和单隐藏层神经网络转移学习的极小化下界