Interpolators are unstable. For example, the mininum $\ell_2$ norm least
square interpolator exhibits unbounded test errors when dealing with noisy
data. In this paper, we study how ensemble stabilizes and thus improves the
generalization performance, measured by the out-of-sample prediction risk, of
an individual interpolator. We focus on bagged linear interpolators, as bagging
is a popular randomization-based ensemble method that can be implemented in
parallel. We introduce the multiplier-bootstrap-based bagged least square
estimator, which can then be formulated as an average of the sketched least
square estimators. The proposed multiplier bootstrap encompasses the classical
bootstrap with replacement as a special case, along with a more intriguing
variant which we call the Bernoulli bootstrap.
Focusing on the proportional regime where the sample size scales
proportionally with the feature dimensionality, we investigate the
out-of-sample prediction risks of the sketched and bagged least square
estimators in both underparametrized and overparameterized regimes. Our results
reveal the statistical roles of sketching and bagging. In particular, sketching
modifies the aspect ratio and shifts the interpolation threshold of the minimum
$\ell_2$ norm estimator. However, the risk of the sketched estimator continues
to be unbounded around the interpolation threshold due to excessive variance.
In stark contrast, bagging effectively mitigates this variance, leading to a
bounded limiting out-of-sample prediction risk. To further understand this
stability improvement property, we establish that bagging acts as a form of
implicit regularization, substantiated by the equivalence of the bagged
estimator with its explicitly regularized counterpart. We also discuss several
extensions.

插值器不稳定，论文研究了集合稳定如何改善插值器的泛化性能，Bagging 作为一种随机化集合方法能够并行实现，通过引入基于乘法自助法的 Bagged 最小二乘估计器，对最小二乘估计器求平均，得到该方法。该文进一步探讨了 Sketching 和 Bagging 对样品内外预测风险的影响，并发现 Bagging 作为一种形式的隐式正则化手段，能够有效减小方差。

集成线性插值器：集成的作用

Ensemble linear interpolators: The role of ensembling

Interpolators -- estimators that achieve zero training error -- have
attracted growing attention in machine learning, mainly because state-of-the
art neural networks appear to be models of this type. In this paper, we study
minimum $\ell_2$ norm ("ridgeless") interpolation in high-dimensional least
squares regression. We consider two different models for the feature
distribution: a linear model, where the feature vectors $x_i \in {\mathbb R}^p$
are obtained by applying a linear transform to a vector of i.i.d. entries, $x_i
= \Sigma^{1/2} z_i$ (with $z_i \in {\mathbb R}^p$); and a nonlinear model,
where the feature vectors are obtained by passing the input through a random
one-layer neural network, $x_i = \varphi(W z_i)$ (with $z_i \in {\mathbb R}^d$,
$W \in {\mathbb R}^{p \times d}$ a matrix of i.i.d. entries, and $\varphi$ an
activation function acting componentwise on $W z_i$). We recover -- in a
precise quantitative way -- several phenomena that have been observed in
large-scale neural networks and kernel machines, including the "double descent"
behavior of the prediction risk, and the potential benefits of
overparametrization.

本文研究了高维最小二乘回归中的最小 L2 范数（“无岭”）插值，并考虑了特征分布的两个不同模型：线性模型和非线性模型