We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method often significantly outperforms existing methods in terms of the training objective, and performs as well or better than optimally-tuned stochastic gradient methods in terms of test error.

我们运用随机平均梯度（SAG）算法来训练条件随机场（CRFs）。我们提出了一种实用的实现方法，利用CRF梯度中的结构来降低这种线性收敛随机梯度方法的内存需求，提出了一种非均匀采样方案，极大地提高了实际性能，并通过非均匀采样分析了SAGA变量的收敛速度。我们的实验结果表明，我们的方法在培训目标方面经常明显优于现有方法，并在测试误差方面表现与经过最佳调整的随机梯度方法一样好甚至更好。

训练条件随机场的非均匀随机平均梯度方法