BriefGPT.xyz
Jun, 2024
随机Polyak步长和动量:收敛保证和实际性能
Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance
HTML
PDF
Dimitris Oikonomou, Nicolas Loizou
TL;DR
在本文中,我们提出了一种基于随机梯度下降算法的新型多步骤选择方法来解决大规模随机优化问题,该方法不需要预先了解问题参数并且具有收敛性保证。
Abstract
stochastic gradient descent
with
momentum
, also known as Stochastic Heavy Ball method (SHB), is one of the most popular algorithms for solving large-scale
→