This paper studies the generalization performance of iterates obtained by Gradient Descent (GD), Stochastic Gradient Descent (SGD) and their proximal variants in high-dimensional robust regression problems. The number of features is comparable to the sample size and errors may be heavy-tailed. We introduce estimators that precisely track the generalization error of the iterates along the trajectory of the iterative algorithm. These estimators are provably consistent under suitable conditions. The results are illustrated through several examples, including Huber regression, pseudo-Huber regression, and their penalized variants with non-smooth regularizer. We provide explicit generalization error estimates for iterates generated from GD and SGD, or from proximal SGD in the presence of a non-smooth regularizer. The proposed risk estimates serve as effective proxies for the actual generalization error, allowing us to determine the optimal stopping iteration that minimizes the generalization error. Extensive simulations confirm the effectiveness of the proposed generalization error estimates.

本文研究了在高维鲁棒回归问题中，通过梯度下降（GD）、随机梯度下降（SGD）及其近端变体获得的迭代结果的泛化性能。通过引入合适条件下可证明一致的估计量，我们提供了明确的泛化误差估计，并有效地确定了最小化泛化误差的最佳停止迭代。

沿着近端随机梯度下降轨迹估计鲁棒回归的泛化性能