In recent years, generated content in music has gained significant
popularity, with large language models being effectively utilized to produce
human-like lyrics in various styles, themes, and linguistic structures. This
technological advancement supports artists in their creative processes but also
raises issues of authorship infringement, consumer satisfaction and content
spamming. To address these challenges, methods for detecting generated lyrics
are necessary. However, existing works have not yet focused on this specific
modality or on creative text in general regarding machine-generated content
detection methods and datasets. In response, we have curated the first dataset
of high-quality synthetic lyrics and conducted a comprehensive quantitative
evaluation of various few-shot content detection approaches, testing their
generalization capabilities and complementing this with a human evaluation. Our
best few-shot detector, based on LLM2Vec, surpasses stylistic and statistical
methods, which are shown competitive in other domains at distinguishing
human-written from machine-generated content. It also shows good generalization
capabilities to new artists and models, and effectively detects post-generation
paraphrasing. This study emphasizes the need for further research on creative
content detection, particularly in terms of generalization and scalability with
larger song catalogs. All datasets, pre-processing scripts, and code are
available publicly on GitHub and Hugging Face under the Apache 2.0 license.

通过对高质量合成歌词进行综合定量评估，我们基于 LIM2 模型构建的最佳 few-shot 检测器，在区分人工创作和机器生成的内容方面超越了样式和统计方法，并且对新艺术家和模型具有良好的泛化能力，同时有效地检测后期生成的改写。

使用少样本推理检测合成歌词

Detecting Synthetic Lyrics with Few-Shot Inference

The potential misuse of ChatGPT and other Large Language Models (LLMs) has
raised concerns regarding the dissemination of false information, plagiarism,
academic dishonesty, and fraudulent activities. Consequently, distinguishing
between AI-generated and human-generated content has emerged as an intriguing
research topic. However, current text detection methods lack precision and are
often restricted to specific tasks or domains, making them inadequate for
identifying content generated by ChatGPT. In this paper, we propose an
effective ChatGPT detector named DEMASQ, which accurately identifies
ChatGPT-generated content. Our method addresses two critical factors: (i) the
distinct biases in text composition observed in human- and machine-generated
content and (ii) the alterations made by humans to evade previous detection
methods. DEMASQ is an energy-based detection model that incorporates novel
aspects, such as (i) optimization inspired by the Doppler effect to capture the
interdependence between input text embeddings and output labels, and (ii) the
use of explainable AI techniques to generate diverse perturbations. To evaluate
our detector, we create a benchmark dataset comprising a mixture of prompts
from both ChatGPT and humans, encompassing domains such as medical, open Q&A,
finance, wiki, and Reddit. Our evaluation demonstrates that DEMASQ achieves
high accuracy in identifying content generated by ChatGPT.

本文提出了一种有效的聊天 GPT 检测器 DEMASQ，能够准确识别由 ChatGPT 生成的内容，并通过基于多普勒效应的优化和可解释的 AI 技术生成多样的扰动，从而解决了人类和机器生成内容的不同偏差以及为了规避先前的检测方法而做出的修改。通过创建涵盖医疗、开放问答、金融、维基和 Reddit 等领域的混合提示的基准数据集，我们的评估表明 DEMASQ 在识别由 ChatGPT 生成的内容方面具有高准确性。

DEMASQ: ChatGPT 字匠的揭开面纱

DEMASQ: Unmasking the ChatGPT Wordsmith

Thanks to the state-of-the-art Large Language Models (LLMs), language
generation has reached outstanding levels. These models are capable of
generating high quality content, thus making it a challenging task to detect
generated text from human-written content. Despite the advantages provided by
Natural Language Generation, the inability to distinguish automatically
generated text can raise ethical concerns in terms of authenticity.
Consequently, it is important to design and develop methodologies to detect
artificial content. In our work, we present some classification models
constructed by ensembling transformer models such as Sci-BERT, DeBERTa and
XLNet, with Convolutional Neural Networks (CNNs). Our experiments demonstrate
that the considered ensemble architectures surpass the performance of the
individual transformer models for classification. Furthermore, the proposed
SciBERT-CNN ensemble model produced an F1-score of 98.36% on the ALTA shared
task 2023 data.

我们的研究通过将 Sci-BERT、DeBERTa 和 XLNet 等变形器模型与卷积神经网络 (CNNs) 进行集成，构建了一些分类模型，实验证明考虑的集成架构在分类方面的性能超过了单个变形器模型。此外，所提出的 SciBERT-CNN 集成模型在 ALTA 共享任务 2023 数据上产生了 98.36% 的 F1-score。