Stochastic algorithms are efficient approaches to solving machine learning and optimization problems. In this paper, we propose a general framework called Splash for parallelizing stochastic algorithms on multi-node distributed systems. Splash consists of a programming interface and an execution engine. Using the programming interface, the user develops sequential stochastic algorithms without concerning any detail about distributed computing. The algorithm is then automatically parallelized by a communication-efficient execution engine. We provide theoretical justifications on the optimal rate of convergence for parallelizing stochastic gradient descent. The real-data experiments with stochastic gradient descent, collapsed Gibbs sampling, stochastic variational inference and stochastic collaborative filtering verify that Splash yields order-of-magnitude speedup over single-thread stochastic algorithms and over parallelized batch algorithms. Besides its efficiency, Splash provides a rich collection of interfaces for algorithm implementation. It is built on Apache Spark and is closely integrated with the Spark ecosystem.

本文提出了一个称为Splash的通用框架，用于在多节点分布式系统上并行化随机算法，该框架以编程接口和执行引擎组成。在实现中，我们提供了关于并行化随机梯度下降的最优收敛率的理论论述。实验结果表明，Splash相对于单线程随机算法和Spark上最先进的实现可以获得数量级的加速。

Splash: 为并行化随机算法提供用户友好的编程接口