With the growing popularity of Large Language Models (e.g. GitHub Copilot,
ChatGPT, etc.) in software engineers' daily practices, it is important to
ensure that the code generated by these tools is not only functionally correct
but also free of vulnerabilities. Although LLMs can help developers to be more
productive, prior empirical studies have shown that LLMs can generate insecure
code. There are two contributing factors to the insecure code generation.
First, existing datasets used to evaluate Large Language Models (LLMs) do not
adequately represent genuine software engineering tasks sensitive to security.
Instead, they are often based on competitive programming challenges or
classroom-type coding tasks. In real-world applications, the code produced is
integrated into larger codebases, introducing potential security risks. There's
a clear absence of benchmarks that focus on evaluating the security of the
generated code. Second, existing evaluation metrics primarily focus on the
functional correctness of the generated code while ignoring security
considerations. Metrics such as pass@k gauge the probability of obtaining the
correct code in the top k suggestions. Other popular metrics like BLEU,
CodeBLEU, ROUGE, and METEOR similarly emphasize functional accuracy, neglecting
security implications. In light of these research gaps, in this paper, we
described SALLM, a framework to benchmark LLMs' abilities to generate secure
code systematically. This framework has three major components: a novel dataset
of security-centric Python prompts, an evaluation environment to test the
generated code, and novel metrics to evaluate the models' performance from the
perspective of secure code generation.

作者描述了一个名为 SALLM 的框架，用于系统地评估大型语言模型生成安全代码的能力，该框架包括一个安全中心的 Python 提示的新数据集，一个用于测试生成代码的评估环境，以及用于从安全代码生成的角度评估模型性能的新度量标准。