Novel artificial intelligence (AI) technology has expedited various
scientific research, e.g., cosmology, physics and bioinformatics, inevitably
becoming a significant category of workload on high performance computing (HPC)
systems. Existing AI benchmarks tend to customize well-recognized AI
applications, so as to evaluate the AI performance of HPC systems under
predefined problem size, in terms of datasets and AI models. Due to lack of
scalability on the problem size, static AI benchmarks might be under competent
to help understand the performance trend of evolving AI applications on HPC
systems, in particular, the scientific AI applications on large-scale systems.
In this paper, we propose a scalable evaluation methodology (SAIH) for
analyzing the AI performance trend of HPC systems with scaling the problem
sizes of customized AI applications. To enable scalability, SAIH builds a set
of novel mechanisms for augmenting problem sizes. As the data and model
constantly scale, we can investigate the trend and range of AI performance on
HPC systems, and further diagnose system bottlenecks. To verify our
methodology, we augment a cosmological AI application to evaluate a real HPC
system equipped with GPUs as a case study of SAIH.

提出了一种可扩展的评估方法（SAIH），用于分析精通于高性能计算系统上的人工智能性能趋势，通过扩大自定义应用程序的问题规模进行评估，从而进一步诊断系统瓶颈。