The imperative need to scale computation across numerous nodes highlights the
significance of efficient parallel computing, particularly in the realm of
Message Passing Interface (MPI) integration. The challenging parallel
programming task of generating MPI-based parallel programs has remained
unexplored. This study first investigates the performance of state-of-the-art
language models in generating MPI-based parallel programs. Findings reveal that
widely used models such as GPT-3.5 and PolyCoder (specialized multi-lingual
code models) exhibit notable performance degradation, when generating MPI-based
programs compared to general-purpose programs. In contrast, domain-specific
models such as MonoCoder, which are pretrained on MPI-related programming
languages of C and C++, outperform larger models. Subsequently, we introduce a
dedicated downstream task of MPI-based program generation by fine-tuning
MonoCoder on HPCorpusMPI. We call the resulting model as MPIrigen. We propose
an innovative preprocessing for completion only after observing the whole code,
thus enabling better completion with a wider context. Comparative analysis
against GPT-3.5 zero-shot performance, using a novel HPC-oriented evaluation
method, demonstrates that MPIrigen excels in generating accurate MPI functions
up to 0.8 accuracy in location and function predictions, and with more than 0.9
accuracy for argument predictions. The success of this tailored solution
underscores the importance of domain-specific fine-tuning in optimizing
language models for parallel computing code generation, paving the way for a
new generation of automatic parallelization tools. The sources of this work are
available at our GitHub MPIrigen repository:
this https URL

MPIrigen 是一个通过领域特定微调为并行计算代码生成优化语言模型的定制解决方案，能够准确生成 0.8 的 MPI 函数位置和功能预测，并且超过 0.9 的参数预测准确率，为自动并行化工具的新一代铺平了道路。