This paper considers the problem of distributed multi-agent learning, where the global aim is to minimize a sum of local objective (empirical loss) functions through local optimization and information exchange between neighbouring nodes. We introduce a Newton-type fully distributed optimization algorithm, Network-GIANT, which is based on GIANT, a Federated learning algorithm that relies on a centralized parameter server. The Network-GIANT algorithm is designed via a combination of gradient-tracking and a Newton-type iterative algorithm at each node with consensus based averaging of local gradient and Newton updates. We prove that our algorithm guarantees semi-global and exponential convergence to the exact solution over the network assuming strongly convex and smooth loss functions. We provide empirical evidence of the superior convergence performance of Network-GIANT over other state-of-art distributed learning algorithms such as Network-DANE and Newton-Raphson Consensus.

介绍了一种基于网络的全新的分布式优化算法Network-GIANT，它使用了类似GIANT的联邦学习算法，基于梯度跟踪和牛顿迭代算法的组合设计，并保证了在强凸光滑损失函数假设下的半全局指数收敛性，相比其他分布式学习算法提高了收敛性能。

Network-GIANT：基于谐波Hessian共识的完全分布式Newton型优化