In this paper, an online learning algorithm is proposed as sequential
stochastic approximation of a regularization path converging to the regression
function in reproducing kernel Hilbert spaces (RKHSs). We show that it is
possible to produce the best known strong (RKHS norm) convergence rate of batch
learning, through a careful choice of the gain or step size sequences,
depending on regularity assumptions on the regression function. The
corresponding weak (mean square distance) convergence rate is optimal in the
sense that it reaches the minimax and individual lower rates in the literature.
In both cases we deduce almost sure convergence, using Bernstein-type
inequalities for martingales in Hilbert spaces.
To achieve this we develop a bias-variance decomposition similar to the batch
learning setting; the bias consists in the approximation and drift errors along
the regularization path, which display the same rates of convergence, and the
variance arises from the sample error analysed as a reverse martingale
difference sequence. The rates above are obtained by an optimal trade-off
between the bias and the variance.

本文介绍一种在线学习算法，该算法是收敛于再生核希尔伯特空间（RKHS）中的回归函数的正则化路径的顺序随机逼近。通过小心选择增益或步长序列，我们展示了可以生产出批量学习的最佳已知强收敛速率，并给出了弱收敛速率，其在文献中达到了最小化和个人较低速率的最优水平，并利用 Hilbert 空间中鞍点型不等式为鞍点型型不等式的马尔可夫过程推导出几乎肯定的收敛。通过类似于批量学习设置的偏差 - 方差分解，我们证明偏差包括沿正则化路径的逼近误差和漂移误差，这些误差显现了相同的收敛速率，而方差则来自样本误差，分析为反向鞍点型差分序列，上述速率通过偏差和方差之间的最佳折衷得到。