Large Language Models (LLMs), such as GPT-3 and BERT, reshape how textual
content is written and communicated. These models have the potential to
generate scientific content that is indistinguishable from that written by
humans. Hence, LLMs carry severe consequences for the scientific community,
which relies on the integrity and reliability of publications. This research
paper presents a novel ChatGPT-generated scientific text detection method,
AI-Catcher. AI-Catcher integrates two deep learning models, multilayer
perceptron (MLP) and convolutional neural networks (CNN). The MLP learns the
feature representations of the linguistic and statistical features. The CNN
extracts high-level representations of the sequential patterns from the textual
content. AI-Catcher is a multimodal model that fuses hidden patterns derived
from MLP and CNN. In addition, a new ChatGPT-Generated scientific text dataset
is collected to enhance AI-generated text detection tools, AIGTxt. AIGTxt
contains 3000 records collected from published academic articles across ten
domains and divided into three classes: Human-written, ChatGPT-generated, and
Mixed text. Several experiments are conducted to evaluate the performance of
AI-Catcher. The comparative results demonstrate the capability of AI-Catcher to
distinguish between human-written and ChatGPT-generated scientific text more
accurately than alternative methods. On average, AI-Catcher improved accuracy
by 37.4%.

聊天 GPT 生成科技文本检测方法 AI-Catcher 融合 MLP 和 CNN 模型，通过对语言和统计特征进行特性学习以及从文本内容中提取顺序模式的高级表示，能更准确地区分人类撰写和 GPT 生成的科技文本，平均提高 37.4% 的准确性。

大型语言模型生成科学内容的深度学习检测方法

Deep Learning Detection Method for Large Language Models-Generated  Scientific Content

This paper presents a novel approach for detecting ChatGPT-generated vs.
human-written text using language models. To this end, we first collected and
released a pre-processed dataset named OpenGPTText, which consists of rephrased
content generated using ChatGPT. We then designed, implemented, and trained two
different models for text classification, using Robustly Optimized BERT
Pretraining Approach (RoBERTa) and Text-to-Text Transfer Transformer (T5),
respectively. Our models achieved remarkable results, with an accuracy of over
97% on the test dataset, as evaluated through various metrics. Furthermore, we
conducted an interpretability study to showcase our model's ability to extract
and differentiate key features between human-written and ChatGPT-generated
text. Our findings provide important insights into the effective use of
language models to detect generated text.

本论文提出了一种使用语言模型检测 ChatGPT 生成的文本与人工编写文本的新方法，并设计、实现和训练了两种不同的文本分类模型，使用了 Robustly Optimized BERT Pretraining Approach（RoBERTa）和 Text-to-Text Transfer Transformer（T5），精度超过 97％，我们还进行了可解释性研究，展示了模型提取和区分人工编写和 ChatGPT 生成文本的关键特征的能力，研究结果为有效使用语言模型检测生成文本提供了重要的见解。