SemEval-2024 Task 8 provides a challenge to detect human-written and
machine-generated text. There are 3 subtasks for different detection scenarios.
This paper proposes a system that mainly deals with Subtask B. It aims to
detect if given full text is written by human or is generated by a specific
Large Language Model (LLM), which is actually a multi-class text classification
task. Our team AISPACE conducted a systematic study of fine-tuning
transformer-based models, including encoderonly, decoder-only and
encoder-decoder models. We compared their performance on this task and
identified that encoder-only models performed exceptionally well. We also
applied a weighted Cross Entropy loss function to address the issue of data
imbalance of different class samples. Additionally, we employed softvoting
strategy over multi-models ensemble to enhance the reliability of our
predictions. Our system ranked top 1 in Subtask B, which sets a
state-of-the-art benchmark for this new challenge.

本研究针对 SemEval-2024 Task 8 提出了一个系统，主要处理 Subtask B，该系统旨在检测给定的全文是由人类撰写还是由特定的大型语言模型生成，这是一个多类文本分类任务。我们团队 AISPACE 对基于 Transformer 的模型进行了系统的微调研究，包括仅编码器、仅解码器和编码器 - 解码器模型。我们比较了它们在该任务上的性能，并发现仅编码器模型表现出色。我们还应用了加权交叉熵损失函数来解决不同类别样本不平衡的问题。此外，我们采用了多模型集成的软投票策略来提高预测的可靠性。我们的系统在 Subtask B 中排名第一，为这个新挑战设立了最新的基准。

AISPACE 在 SemEval-2024 任务 8 上：用于检测多生成器机器生成文本的类平衡软投票系统

AISPACE at SemEval-2024 task 8: A Class-balanced Soft-voting System for  Detecting Multi-generator Machine-generated Text

SemEval-2024 Task 8 introduces the challenge of identifying machine-generated
texts from diverse Large Language Models (LLMs) in various languages and
domains. The task comprises three subtasks: binary classification in
monolingual and multilingual (Subtask A), multi-class classification (Subtask
B), and mixed text detection (Subtask C). This paper focuses on Subtask A & B.
Each subtask is supported by three datasets for training, development, and
testing. To tackle this task, two methods: 1) using traditional machine
learning (ML) with natural language preprocessing (NLP) for feature extraction,
and 2) fine-tuning LLMs for text classification. The results show that
transformer models, particularly LoRA-RoBERTa, exceed traditional ML methods in
effectiveness, with majority voting being particularly effective in
multilingual contexts for identifying machine-generated texts.

SemEval-2024 Task 8 引入了识别多语言和领域中大型语言模型（LLMs）生成的机器文本的挑战。本文关注于二元分类和多类分类的子任务，并通过传统机器学习和自然语言处理进行特征提取以及基于 LLMs 进行文本分类的两种方法进行解决，结果显示变压器模型，尤其是 LoRA-RoBERTa，在多语言环境中使用多数表决方法能够高效地识别机器生成的文本。