In this paper we analyze features to classify human- and AI-generated text
for English, French, German and Spanish and compare them across languages. We
investigate two scenarios: (1) The detection of text generated by AI from
scratch, and (2) the detection of text rephrased by AI. For training and
testing the classifiers in this multilingual setting, we created a new text
corpus covering 10 topics for each language. For the detection of AI-generated
text, the combination of all proposed features performs best, indicating that
our features are portable to other related languages: The F1-scores are close
with 99% for Spanish, 98% for English, 97% for German and 95% for French. For
the detection of AI-rephrased text, the systems with all features outperform
systems with other features in many cases, but using only document features
performs best for German (72%) and Spanish (86%) and only text vector features
leads to best results for English (78%).

本文分析了用于英语、法语、德语和西班牙语识别人工智能生成文本和人类生成文本的特征，并对不同语言进行了比较。研究了两种情况：(1) 检测由人工智能生成的原创文本，和 (2) 检测由人工智能重新表述的文本。通过建立包含每种语言 10 个主题的新文本语料库来训练和测试这个多语言环境中的分类器。特征的组合在检测人工智能生成的文本时表现最好，说明这些特征可以应用于其他相关语言：西班牙语的 F1 得分接近 99%，英语为 98%，德语为 97%，法语为 95%。在检测人工智能重新表述的文本时，大多数情况下使用所有特征的系统效果最好，但对于德语（72%）和西班牙语（86%），仅使用文档特征最好，而对英语则仅使用文本向量特征可获得最佳结果（78%）。

英法德西语人工智能与人类生成文本分类

Classification of Human- and AI-Generated Texts for English, French,  German, and Spanish

We propose an alternate approach to quantifying how well language models
learn natural language: we ask how well they match the statistical tendencies
of natural language. To answer this question, we analyze whether text generated
from language models exhibits the statistical tendencies present in the
human-generated text on which they were trained. We provide a framework--paired
with significance tests--for evaluating the fit of language models to these
trends. We find that neural language models appear to learn only a subset of
the tendencies considered, but align much more closely with empirical trends
than proposed theoretical distributions (when present). Further, the fit to
different distributions is highly-dependent on both model architecture and
generation strategy. As concrete examples, text generated under the nucleus
sampling scheme adheres more closely to the type--token relationship of natural
language than text produced using standard ancestral sampling; text from LSTMs
reflects the natural language distributions over length, stopwords, and symbols
surprisingly well.

我们提出了一个替代方法来量化语言模型学习自然语言的程度：我们询问它们多大程度上与自然语言的统计倾向相匹配。通过分析语言模型生成的文本是否呈现出它们所训练的人类生成文本中存在的统计倾向，提供了一个与显著性测试配对的框架来评估语言模型的拟合程度。我们发现神经语言模型似乎只学会了一部分被考虑的倾向，但与经验性趋势相比，更接近所提出的理论分布（当存在时）。此外，对不同分布的拟合程度高度依赖于模型架构和生成策略。作为具体例子，使用 Nucleus sampling 方式生成的文本比使用标准祖先抽样生成的文本更紧密地遵循自然语言的类型 - 标记关系；LSTMs 生成的文本也非常好地反映了自然语言的长度、停用词和符号分布。