Deep learning research has uncovered the phenomenon of benign overfitting for
over-parameterized statistical models, which has drawn significant theoretical
interest in recent years. Given its simplicity and practicality, the ordinary
least squares (OLS) interpolator has become essential to gain foundational
insights into this phenomenon. While properties of OLS are well established in
classical settings, its behavior in high-dimensional settings is less explored
(unlike for ridge or lasso regression) though significant progress has been
made of late. We contribute to this growing literature by providing fundamental
algebraic and statistical results for the minimum $\ell_2$-norm OLS
interpolator. In particular, we provide high-dimensional algebraic equivalents
of (i) the leave-$k$-out residual formula, (ii) Cochran's formula, and (iii)
the Frisch-Waugh-Lovell theorem. These results aid in understanding the OLS
interpolator's ability to generalize and have substantive implications for
causal inference. Additionally, under the Gauss-Markov model, we present
statistical results such as a high-dimensional extension of the Gauss-Markov
theorem and an analysis of variance estimation under homoskedastic errors. To
substantiate our theoretical contributions, we conduct simulation studies that
further explore the stochastic properties of the OLS interpolator.

通过提供 OLS 插值器的高维代数和统计结果，我们对其一般化能力和因果推断具有实质性影响进行了研究，此外，我们还在高斯 - 马尔可夫模型下提出了统计结果和方差估计的分析。