This paper proposes an asymptotic theory for online inference of the stochastic gradient descent (SGD) iterates with dropout regularization in linear regression. Specifically, we establish the geometric-moment contraction (GMC) for constant step-size SGD dropout iterates to show the existence of a unique stationary distribution of the dropout recursive function. By the GMC property, we provide quenched central limit theorems (CLT) for the difference between dropout and $\ell^2$-regularized iterates, regardless of initialization. The CLT for the difference between the Ruppert-Polyak averaged SGD (ASGD) with dropout and $\ell^2$-regularized iterates is also presented. Based on these asymptotic normality results, we further introduce an online estimator for the long-run covariance matrix of ASGD dropout to facilitate inference in a recursive manner with efficiency in computational time and memory. The numerical experiments demonstrate that for sufficiently large samples, the proposed confidence intervals for ASGD with dropout nearly achieve the nominal coverage probability.

本文研究了线性回归中带Dropout正则化的随机梯度下降（SGD）迭代的渐近理论，填补了现有文献对此问题的理论空白。通过建立几何矩收缩性（GMC），作者展示了Dropout递归函数存在唯一的平稳分布，并提出了针对ASGD Dropout的在线协方差矩阵估计方法，为后续递归推断提供了高效的计算框架。实验结果表明，该方法在大样本情况下的置信区间几乎达到了名义覆盖概率。 

带Dropout正则化的随机梯度下降的渐近理论研究