BriefGPT.xyz
Jun, 2021
随机凸优化中不要使用完整批次
Never Go Full Batch (in Stochastic Convex Optimization)
HTML
PDF
Idan Amir, Yair Carmon, Tomer Koren, Roi Livni
TL;DR
本文研究随机凸优化的全批量优化算法的泛化性能,并提供了一项新的分离结果,显示出全批量方法需要至少Ω(1 /ε^4)次迭代或具有维度相关的样本复杂度。
Abstract
We study the
generalization performance
of $\text{full-batch}$ optimization algorithms for stochastic
convex optimization
: these are first-order methods that only access the exact gradient of the empirical risk (
→