BriefGPT.xyz
Jun, 2024
开放问题：梯度下降的逐步收敛速度
Open Problem: Anytime Convergence Rate of Gradient Descent
HTML
PDF
Guy Kornowski, Ohad Shamir
TL;DR
通过改变步长序列，可以加速原始的梯度下降方法，并导致不断增大的误差，因此我们提出了一个问题：是否存在可以在任意停止时间下加速经典的 $\mathcal{O}(1/T)$ 收敛速度的梯度下降步长安排？
Abstract
Recent results show that vanilla
gradient descent
can be accelerated for smooth convex objectives, merely by changing the
stepsize sequence
. We show that this can lead to surprisingly large errors indefinitely, a
→