While distributed training is often viewed as a solution to optimizing linear
models on increasingly large datasets, inter-machine communication costs of
popular distributed approaches can dominate as data dimensionality increases.
Recent work on non-interactive algorithms shows that approximate solutions for
linear models can be obtained efficiently with only a single round of
communication among machines. However, this approximation often degenerates as
the number of machines increases. In this paper, building on the recent optimal
weighted average method, we introduce a new technique, ACOWA, that allows an
extra round of communication to achieve noticeably better approximation quality
with minor runtime increases. Results show that for sparse distributed logistic
regression, ACOWA obtains solutions that are more faithful to the empirical
risk minimizer and attain substantially higher accuracy than other distributed
algorithms.

通过 ACOWA 技术，在分布式训练中，通过额外的通信轮次获得更准确的近似解，相较于其他分布式算法，对于稀疏分布式逻辑回归，ACOWA 获得更接近经验风险最小化的解并实现更高的准确性。