mini-batch stochastic gradient descent (SGD) is the state of the art in large scale parallel machine learning, but its scalability is limited by a communication bottleneck. Recent work proposed local sgd, i.e. ru