Transfer learning is a critical part of real-world machine learning
deployments and has been extensively studied in experimental works with
overparameterized neural networks. However, even in the simplest setting of
linear regression a notable gap still exists in the theoretical understanding
of transfer learning. In-distribution research on high-dimensional linear
regression has led to the identification of a phenomenon known as
\textit{benign overfitting}, in which linear interpolators overfit to noisy
training labels and yet still generalize well. This behavior occurs under
specific conditions on the source covariance matrix and input data dimension.
Therefore, it is natural to wonder how such high-dimensional linear models
behave under transfer learning. We prove the first non-asymptotic excess risk
bounds for benignly-overfit linear interpolators in the transfer learning
setting. From our analysis, we propose a taxonomy of \textit{beneficial} and
\textit{malignant} covariate shifts based on the degree of
overparameterization. We follow our analysis with empirical studies that show
these beneficial and malignant covariate shifts for linear interpolators on
real image data, and for fully-connected neural networks in settings where the
input data dimension is larger than the training sample size.

高维线性回归中的过拟合与泛化问题在转移学习中的不同表现及风险边界进行了分析和研究，并提出了基于超参数化程度的有益和有害转变偏差的分类方法。