BriefGPT.xyz
Oct, 2021
现代误差反馈的实用算法扩展 - EF21升级版
EF21 with Bells & Whistles: Practical Algorithmic Extensions of Modern Error Feedback
HTML
PDF
Ilyas Fatkhullin, Igor Sokolov, Eduard Gorbunov, Zhize Li, Peter Richtárik
TL;DR
本文提出了6种有效的实用扩展EF21,并分别分析它们的强收敛理论,这些扩展可以与EF一起使用,采用压缩策略和带有Markov压缩器的新的错误反馈机制可以更好地优化分布式梯度下降算法。
Abstract
First proposed by Seide (2014) as a heuristic,
error feedback
(EF) is a very popular mechanism for enforcing convergence of distributed gradient-based
optimization methods
enhanced with communication
→