It has been shown that finetuned transformers and other supervised detectors
effectively distinguish between human and machine-generated text in some
situations arXiv:2305.13242, but we find that even simple classifiers on top of
n-gram and part-of-speech features can achieve very robust performance on both
in- and out-of-domain data. To understand how this is possible, we analyze
machine-generated output text in five datasets, finding that LLMs possess
unique fingerprints that manifest as slight differences in the frequency of
certain lexical and morphosyntactic features. We show how to visualize such
fingerprints, describe how they can be used to detect machine-generated text
and find that they are even robust across textual domains. We find that
fingerprints are often persistent across models in the same model family (e.g.
llama-13b vs. llama-65b) and that models fine-tuned for chat are easier to
detect than standard language models, indicating that LLM fingerprints may be
directly induced by the training data.

通过分析 LLM 指纹，我们展示了即使是简单的分类器也能在训练领域内外的数据中实现非常强大的性能，从而有效区分人类生成的文本和机器生成的文本。