We analyze the statistical properties of generalized cross-validation (GCV) and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient descent (GD) in high-dimensional least squares regression. We prove that GCV is generically inconsistent as an estimator of the prediction risk of early-stopped GD, even for a well-specified linear model with isotropic features. In contrast, we show that LOOCV converges uniformly along the GD trajectory to the prediction risk. Our theory requires only mild assumptions on the data distribution and does not require the underlying regression function to be linear. Furthermore, by leveraging the individual LOOCV errors, we construct consistent estimators for the entire prediction error distribution along the GD trajectory and consistent estimators for a wide class of error functionals. This in particular enables the construction of pathwise prediction intervals based on GD iterates that have asymptotically correct nominal coverage conditional on the training data.

分析了在高维最小二乘回归中应用的广义交叉验证（GCV）和留一交叉验证（LOOCV）的统计特性，证明了GCV作为早期停止梯度下降的预测风险估计量普遍不一致，而LOOCV则沿着梯度下降轨迹一致收敛于预测风险。利用个别LOOCV误差，构建了整个梯度下降轨迹上预测误差分布的一致估计量和广泛的误差函数的一致估计量，这特别使得基于GD迭代的路径预测区间在培训数据条件下具有渐近正确的名义覆盖率。

早停梯度下降的交叉验证的成功和失败