The use of machine learning (ML) models to assess and score textual data has
become increasingly pervasive in an array of contexts including natural
language processing, information retrieval, search and recommendation, and
credibility assessment of online content. A significant disruption at the
intersection of ML and text are text-generating large-language models such as
generative pre-trained transformers (GPTs). We empirically assess the
differences in how ML-based scoring models trained on human content assess the
quality of content generated by humans versus GPTs. To do so, we propose an
analysis framework that encompasses essay scoring ML-models, human and
ML-generated essays, and a statistical model that parsimoniously considers the
impact of type of respondent, prompt genre, and the ML model used for
assessment model. A rich testbed is utilized that encompasses 18,460
human-generated and GPT-based essays. Results of our benchmark analysis reveal
that transformer pretrained language models (PLMs) more accurately score human
essay quality as compared to CNN/RNN and feature-based ML methods.
Interestingly, we find that the transformer PLMs tend to score GPT-generated
text 10-15\% higher on average, relative to human-authored documents.
Conversely, traditional deep learning and feature-based ML models score human
text considerably higher. Further analysis reveals that although the
transformer PLMs are exclusively fine-tuned on human text, they more
prominently attend to certain tokens appearing only in GPT-generated text,
possibly due to familiarity/overlap in pre-training. Our framework and results
have implications for text classification settings where automated scoring of
text is likely to be disrupted by generative AI.

基于机器学习模型对文本数据进行评分的使用已广泛应用于自然语言处理、信息检索、搜索和推荐以及在线内容的可信度评估等领域。这项研究通过实证评估以人工创作和生成预训练变换器（GPT）的文本评估模型之间的差异，发现转换器预训练语言模型（PLM）相对于传统的深度学习和基于特征的机器学习模型更准确地评分人工文本质量，但相对于人工创作的文档，GPT 生成的文本评分平均要高出 10-15％。这一研究对于文本分类设置中的自动评分受到生成 AI 的干扰具有重要的意义。

当自动评估遇上自动内容生成：审视 GPT 时代的文本质量

When Automated Assessment Meets Automated Content Generation: Examining  Text Quality in the Era of GPTs

This paper presents an investigation of using a co-attention based neural
network for source-dependent essay scoring. We use a co-attention mechanism to
help the model learn the importance of each part of the essay more accurately.
Also, this paper shows that the co-attention based neural network model
provides reliable score prediction of source-dependent responses. We evaluate
our model on two source-dependent response corpora. Results show that our model
outperforms the baseline on both corpora. We also show that the attention of
the model is similar to the expert opinions with examples.

本文报道了使用基于共同注意力的神经网络进行依赖于来源的文章评分的研究。研究结果表明，该模型提供了可靠的依赖于来源的文章评分预测，并在两个依赖于来源的响应语料库上优于基准模型，而且其注意力与专家意见相似，具有高度的准确性。