Educators are increasingly concerned about the usage of Large Language Models
(LLMs) such as ChatGPT in programming education, particularly regarding the
potential exploitation of imperfections in Artificial Intelligence Generated
Content (AIGC) Detectors for academic misconduct. In this paper, we present an
empirical study where the LLM is examined for its attempts to bypass detection
by AIGC Detectors. This is achieved by generating code in response to a given
question using different variants. We collected a dataset comprising 5,069
samples, with each sample consisting of a textual description of a coding
problem and its corresponding human-written Python solution codes. These
samples were obtained from various sources, including 80 from Quescol, 3,264
from Kaggle, and 1,725 from LeetCode. From the dataset, we created 13 sets of
code problem variant prompts, which were used to instruct ChatGPT to generate
the outputs. Subsequently, we assessed the performance of five AIGC detectors.
Our results demonstrate that existing AIGC Detectors perform poorly in
distinguishing between human-written code and AI-generated code.

研究聚焦于大型语言模型在编程教育中的应用，特别关注人工智能生成内容检测器在学术不端中的潜在漏洞，并通过生成代码来检验大型语言模型对于绕过检测的努力。研究结果表明现有的人工智能生成内容检测器在区别人工编写的代码和人工智能生成的代码方面表现不佳。

评估 AI 检测器在识别 AI 生成代码中的应用：对教育的影响

Assessing AI Detectors in Identifying AI-Generated Code: Implications  for Education

Large language models (LLMs) have opened up enormous opportunities while
simultaneously posing ethical dilemmas. One of the major concerns is their
ability to create text that closely mimics human writing, which can lead to
potential misuse, such as academic misconduct, disinformation, and fraud. To
address this problem, we present AuthentiGPT, an efficient classifier that
distinguishes between machine-generated and human-written texts. Under the
assumption that human-written text resides outside the distribution of
machine-generated text, AuthentiGPT leverages a black-box LLM to denoise input
text with artificially added noise, and then semantically compares the denoised
text with the original to determine if the content is machine-generated. With
only one trainable parameter, AuthentiGPT eliminates the need for a large
training dataset, watermarking the LLM's output, or computing the
log-likelihood. Importantly, the detection capability of AuthentiGPT can be
easily adapted to any generative language model. With a 0.918 AUROC score on a
domain-specific dataset, AuthentiGPT demonstrates its effectiveness over other
commercial algorithms, highlighting its potential for detecting
machine-generated text in academic settings.

鉴于大型语言模型（LLMs）在同时提供巨大机遇的同时也引发了伦理困境，我们提出了 AuthentiGPT，这是一个有效的分类器，用于区分机器生成的和人类编写的文本。通过给输入文本添加人工噪声，并在语义上比较去噪后的文本与原始文本，AuthentiGPT 利用了黑盒 LLM 来消除输入文本的噪声，以判断内容是否为机器生成。通过只有一个可训练参数，AuthentiGPT 消除了需要大量训练数据集、对 LLM 输出进行水印处理或计算对数似然的需求。在特定领域数据集上具有 0.918 AUROC 分数的 AuthentiGPT 表明其在检测学术环境中的机器生成文本方面的有效性，并凸显了它的潜力。