The freedom of fast iterations of distributed deep learning tasks is crucial
for smaller companies to gain competitive advantages and market shares from big
tech giants. HorovodRunner brings this process to relatively accessible spark
clusters. There have been, however, no benchmark tests on HorovodRunner per se,
nor specifically graph convolutional network (GCN, hereafter), and very limited
scalability benchmark tests on Horovod, the predecessor requiring custom built
GPU clusters. For the first time, we show that Databricks' HorovodRunner
achieves significant lift in scaling efficiency for the convolutional neural
network (CNN, hereafter) based tasks on both GPU and CPU clusters, but not the
original GCN task. We also implemented the Rectified Adam optimizer for the
first time in HorovodRunner.

本文首次展示了 Databricks 的 HorovodRunner 在 GPU 和 CPU 集群上对基于卷积神经网络（CNN）的任务具有显著的扩展效率提升，但并没有在原始 GCN 任务上实现相同的效果。此外，我们还首次在 HorovodRunner 中实现了修正的 Adam 优化算法。