The growing prominence of large language models, such as GPT-4 and ChatGPT,
has led to increased concerns over academic integrity due to the potential for
machine-generated content and paraphrasing. Although studies have explored the
detection of human- and machine-paraphrased content, the comparison between
these types of content remains underexplored. In this paper, we conduct a
comprehensive analysis of various datasets commonly employed for paraphrase
detection tasks and evaluate an array of detection methods. Our findings
highlight the strengths and limitations of different detection methods in terms
of performance on individual datasets, revealing a lack of suitable
machine-generated datasets that can be aligned with human expectations. Our
main finding is that human-authored paraphrases exceed machine-generated ones
in terms of difficulty, diversity, and similarity implying that automatically
generated texts are not yet on par with human-level performance. Transformers
emerged as the most effective method across datasets with TF-IDF excelling on
semantically diverse corpora. Additionally, we identify four datasets as the
most diverse and challenging for paraphrase detection.

本文旨在分析大型语言模型如 GPT-4 和 ChatGPT 等的语言生成和改写能力的特点和限制及其对学术诚信的挑战，探讨人工改写和机器改写之间的区别，并对常用的数据集进行综合分析和评估，结果显示自动生成文本的水平仍不及人类普及水平，适合于语义多样性语料的 TF-IDF 方法的效果最好，同时，发现了四个语义多样性和挑战性最高的数据集。