Interactive massively parallel computations are critical for machine learning
and data analysis. These computations are a staple of the MIT Lincoln
Laboratory Supercomputing Center (LLSC) and has required the LLSC to develop
unique interactive supercomputing capabilities. Scaling interactive machine
learning frameworks, such as TensorFlow, and data analysis environments, such
as MATLAB/Octave, to tens of thousands of cores presents many technical
challenges - in particular, rapidly dispatching many tasks through a scheduler,
such as Slurm, and starting many instances of applications with thousands of
dependencies. Careful tuning of launches and prepositioning of applications
overcome these challenges and allow the launching of thousands of tasks in
seconds on a 40,000-core supercomputer. Specifically, this work demonstrates
launching 32,000 TensorFlow processes in 4 seconds and launching 262,000 Octave
processes in 40 seconds. These capabilities allow researchers to rapidly
explore novel machine learning architecture and data analysis algorithms.

本研究展示了如何通过调整启动方法和提前部署应用程序，克服任务调度和依赖关系等技术挑战，使用 LLSC 的大规模超级计算机在几秒内启动数千个任务，例如 32,000 个 TensorFlow 进程和 262,000 个 Octave 进程，进而快速研究新型机器学习架构和数据分析算法。