We evaluate the use of the open-source Llama-2 model for generating
well-known, high-performance computing kernels (e.g., AXPY, GEMV, GEMM) on
different parallel programming models and languages (e.g., C++: OpenMP, OpenMP
Offload, OpenACC, CUDA, HIP; Fortran: OpenMP, OpenMP Offload, OpenACC; Python:
numpy, Numba, pyCUDA, cuPy; and Julia: Threads, CUDA.jl, AMDGPU.jl). We built
upon our previous work that is based on the OpenAI Codex, which is a descendant
of GPT-3, to generate similar kernels with simple prompts via GitHub Copilot.
Our goal is to compare the accuracy of Llama-2 and our original GPT-3 baseline
by using a similar metric. Llama-2 has a simplified model that shows
competitive or even superior accuracy. We also report on the differences
between these foundational large language models as generative AI continues to
redefine human-computer interactions. Overall, Copilot generates codes that are
more reliable but less optimized, whereas codes generated by Llama-2 are less
reliable but more optimized when correct.

我们评估了开源 Llama-2 模型在不同并行编程模型和语言上生成众所周知的高性能计算内核（例如 AXPY、GEMV、GEMM）的使用情况。我们使用简单的提示通过 GitHub Copilot 基于 OpenAI Codex 生成类似的内核，以比较 Llama-2 和我们原始的 GPT-3 基准的准确性，并报告了基于这些基础的大型语言模型的区别，因为生成式人工智能继续重新定义人机交互。总体而言，Copilot 生成的代码更可靠但不太优化，而 Llama-2 生成的代码在正确时更优化但不太可靠。