This paper presents the first, 15-PetaFLOP Deep Learning system for solving scientific pattern classification problems on contemporary HPC architectures. We develop supervised convolutional architectures for discriminating signals in high-energy physics data as well as semi-supervised architectures for localizing and classifying extreme weather in climate data. Our Intelcaffe-based implementation obtains $\sim$2TFLOP/s on a single Cori Phase-II Xeon-Phi node. We use a hybrid strategy employing synchronous node-groups, while using asynchronous communication across groups. We use this strategy to scale training of a single model to $\sim$9600 Xeon-Phi nodes; obtaining peak performance of 11.73-15.07 PFLOP/s and sustained performance of 11.41-13.27 PFLOP/s. At scale, our HEP architecture produces state-of-the-art classification accuracy on a dataset with 10M images, exceeding that achieved by selections on high-level physics-motivated features. Our semi-supervised architecture successfully extracts weather patterns in a 15TB climate dataset. Our results demonstrate that Deep Learning can be optimized and scaled effectively on many-core, HPC systems.

这篇论文介绍了第一个使用现代高性能计算架构解决科学模式分类问题的15-PetaFLOP深度学习系统。通过使用卷积神经网络对高能物理数据和气候数据进行监督和半监督建模，实现了在单个Cori Phase-II Xeon-Phi节点上达到2 TFLOP/s；在使用节点组之间的异步通信和同步节点组的混合策略的情况下，将单模型的训练扩展到9600个Xeon-Phi节点，达到了11.73-15.07 PFLOP/s峰值性能和11.41-13.27 PFLOP/s持续性能。最终，作者的高能物理和气候学模式分类问题的卷积神经网络表现出了超过那些高层次物理学特征选择所达到的最先进的分类准确性；而他们的半监督架构成功地在15TB的气候数据集中提取了天气模式，证明了深度学习在众核HPC系统上的有效优化和扩展.

15PF的深度学习: 面向科学数据的监督和半监督分类