We analyze the statistical properties of generalized cross-validation (GCV)
and leave-one-out cross-validation (LOOCV) applied to early-stopped gradient
descent (GD) in high-dimensional least squares regression. We prove that GCV is
generically inconsistent as an estimator of the prediction risk of
early-stopped GD, even for a well-specified linear model with isotropic
features. In contrast, we show that LOOCV converges uniformly along the GD
trajectory to the prediction risk. Our theory requires only mild assumptions on
the data distribution and does not require the underlying regression function
to be linear. Furthermore, by leveraging the individual LOOCV errors, we
construct consistent estimators for the entire prediction error distribution
along the GD trajectory and consistent estimators for a wide class of error
functionals. This in particular enables the construction of pathwise prediction
intervals based on GD iterates that have asymptotically correct nominal
coverage conditional on the training data.

分析了在高维最小二乘回归中应用的广义交叉验证（GCV）和留一交叉验证（LOOCV）的统计特性，证明了 GCV 作为早期停止梯度下降的预测风险估计量普遍不一致，而 LOOCV 则沿着梯度下降轨迹一致收敛于预测风险。利用个别 LOOCV 误差，构建了整个梯度下降轨迹上预测误差分布的一致估计量和广泛的误差函数的一致估计量，这特别使得基于 GD 迭代的路径预测区间在培训数据条件下具有渐近正确的名义覆盖率。