Recently, generative AIs like ChatGPT have become available to the wide
public. These tools can for instance be used by students to generate essays or
whole theses. But how does a teacher know whether a text is written by a
student or an AI? In our work, we explore traditional and new features to (1)
detect text generated by AI from scratch and (2) text rephrased by AI. Since we
found that classification is more difficult when the AI has been instructed to
create the text in a way that a human would not recognize that it was generated
by an AI, we also investigate this more advanced case. For our experiments, we
produced a new text corpus covering 10 school topics. Our best systems to
classify basic and advanced human-generated/AI-generated texts have F1-scores
of over 96%. Our best systems for classifying basic and advanced
human-generated/AI-rephrased texts have F1-scores of more than 78%. The systems
use a combination of perplexity, semantic, list lookup, error-based,
readability, AI feedback, and text vector features. Our results show that the
new features substantially help to improve the performance of many classifiers.
Our best basic text rephrasing detection system even outperforms GPTZero by
183.8% relative in F1-score.

我们的研究旨在探索传统和新的特征，以便 (1) 检测 AI 生成的文本和 (2) 由 AI 改写的文本。结果显示，新特征显著提高了许多分类器的性能。我们最好的基本文本改写检测系统在 F1 分数上超过了 GPTZero 的 183.8%。