AI-based code generators have gained a fundamental role in assisting
developers in writing software starting from natural language (NL). However,
since these large language models are trained on massive volumes of data
collected from unreliable online sources (e.g., GitHub, Hugging Face), AI
models become an easy target for data poisoning attacks, in which an attacker
corrupts the training data by injecting a small amount of poison into it, i.e.,
astutely crafted malicious samples. In this position paper, we address the
security of AI code generators by identifying a novel data poisoning attack
that results in the generation of vulnerable code. Next, we devise an extensive
evaluation of how these attacks impact state-of-the-art models for code
generation. Lastly, we discuss potential solutions to overcome this threat.

基于人工智能的代码生成器在帮助开发人员从自然语言中编写软件方面起到了重要作用。本文提出了一种新颖的数据污染攻击，其影响是生成易受攻击的代码。我们对这些攻击对代码生成的最新模型的影响进行了广泛评估，并讨论了潜在的解决方案。

AI 生成代码的安全隐患：通过取消修复代码来破坏程序

Poisoning Programs by Un-Repairing Code: Security Concerns of  AI-generated Code

AI-based code generators are an emerging solution for automatically writing
programs starting from descriptions in natural language, by using deep neural
networks (Neural Machine Translation, NMT). In particular, code generators have
been used for ethical hacking and offensive security testing by generating
proof-of-concept attacks. Unfortunately, the evaluation of code generators
still faces several issues. The current practice uses output similarity
metrics, i.e., automatic metrics that compute the textual similarity of
generated code with ground-truth references. However, it is not clear what
metric to use, and which metric is most suitable for specific contexts.
This work analyzes a large set of output similarity metrics on offensive code
generators. We apply the metrics on two state-of-the-art NMT models using two
datasets containing offensive assembly and Python code with their descriptions
in the English language. We compare the estimates from the automatic metrics
with human evaluation and provide practical insights into their strengths and
limitations.

该研究分析了在 AI 代码生成器上采用不同相似度度量方式的适用性，比较了自动估计和人工评估之间的差异，并提供了实用性见解。