BriefGPT.xyz
May, 2017
TernGrad: 用三值梯度减少分布式深度学习通信
TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning
HTML
PDF
Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang...
TL;DR
本文提出TernGrad，使用三元梯度来加速数据并行的分布式深度学习，结果表明可在不损失准确率的前提下缩减通信时间并提高收敛性能，使得多种深度神经网络的速度显著提升。
Abstract
High network communication cost for synchronizing gradients and parameters is the well-known bottleneck of
distributed training
. In this work, we propose TernGrad that uses
ternary gradients
to accelerate distrib
→