Quantum programs are typically developed using quantum Software Development
Kits (SDKs). The rapid advancement of quantum computing necessitates new tools
to streamline this development process, and one such tool could be Generative
Artificial intelligence (GenAI). In this study, we introduce and use the Qiskit
HumanEval dataset, a hand-curated collection of tasks designed to benchmark the
ability of Large Language Models (LLMs) to produce quantum code using Qiskit -
a quantum SDK. This dataset consists of more than 100 quantum computing tasks,
each accompanied by a prompt, a canonical solution, a comprehensive test case,
and a difficulty scale to evaluate the correctness of the generated solutions.
We systematically assess the performance of a set of LLMs against the Qiskit
HumanEval dataset's tasks and focus on the models ability in producing
executable quantum code. Our findings not only demonstrate the feasibility of
using LLMs for generating quantum code but also establish a new benchmark for
ongoing advancements in the field and encourage further exploration and
development of GenAI-driven tools for quantum code generation.

通过使用 Quantum Software Development Kits (SDKs) 进行开发，本研究引入并使用了 Qiskit HumanEval 数据集，这是一个手工精选的任务集合，旨在评估 Large Language Models (LLMs) 在使用 Qiskit（一种量子 SDK）生成量子代码方面的能力，并在可执行的量子代码生成方面建立了新的基准。