This work provides a simplified proof of the statistical minimax optimality
of (iterate averaged) stochastic gradient descent (SGD), for the special case
of least squares. This result is obtained by analyzing SGD as a stochastic
process and by sharply characterizing the stationary covariance matrix of this
process. The finite rate optimality characterization captures the constant
factors and addresses model mis-specification.

本文针对最小二乘法，提供了一个简化的证明，证明了随机梯度下降法（SGD）的（迭代平均）统计极小值最优性，并通过分析 SGD 作为随机过程、对该过程的稳态协方差矩阵进行尖锐刻画等方式进行了证明。有限速率的最优性刻画捕获了常数因子，并解决了模型错误规格化的问题。

马尔科夫链理论方法描述随机梯度下降（针对最小二乘法）的极小极大优化特性

A Markov Chain Theory Approach to Characterizing the Minimax Optimality  of Stochastic Gradient Descent (for Least Squares)

Likelihood based-learning of graphical models faces challenges of
computational-complexity and robustness to model mis-specification. This paper
studies methods that fit parameters directly to maximize a measure of the
accuracy of predicted marginals, taking into account both model and inference
approximations at training time. Experiments on imaging problems suggest
marginalization-based learning performs better than likelihood-based
approximations on difficult problems where the model being fit is approximate
in nature.

本文研究直接拟合参数以最大化预测边际准确性度量的方法，该方法考虑训练时间模型和推理近似。对成像问题的实验表明，边际化学习在拟合的模型近似的困难问题上比似然近似表现更好。