With the launch of ChatGPT, large language models (LLMs) have attracted
global attention. In the realm of article writing, LLMs have witnessed
extensive utilization, giving rise to concerns related to intellectual property
protection, personal privacy, and academic integrity. In response, AI-text
detection has emerged to distinguish between human and machine-generated
content. However, recent research indicates that these detection systems often
lack robustness and struggle to effectively differentiate perturbed texts.
Currently, there is a lack of systematic evaluations regarding detection
performance in real-world applications, and a comprehensive examination of
perturbation techniques and detector robustness is also absent. To bridge this
gap, our work simulates real-world scenarios in both informal and professional
writing, exploring the out-of-the-box performance of current detectors.
Additionally, we have constructed 12 black-box text perturbation methods to
assess the robustness of current detection models across various perturbation
granularities. Furthermore, through adversarial learning experiments, we
investigate the impact of perturbation data augmentation on the robustness of
AI-text detectors. We have released our code and data at
this https URL

利用机器学习模型对文章进行自动生成的大语言模型，在知识产权保护、个人隐私和学术诚信方面引发了关注。AI 文本检测系统目前存在鲁棒性不足和难以有效区分扰动文本的问题。本研究通过模拟真实场景，评估当前检测模型在实际应用中的性能，并构建了 12 种黑盒文本扰动方法来评估检测模型的鲁棒性。此外，通过对抗学习实验，研究扰动数据增强对 AI 文本检测器鲁棒性的影响。