Misinformation is considered a threat to our democratic values and
principles. The spread of such content on social media polarizes society and
undermines public discourse by distorting public perceptions and generating
social unrest while lacking the rigor of traditional journalism. Transformers
and transfer learning proved to be state-of-the-art methods for multiple
well-known natural language processing tasks. In this paper, we propose
MisRoB{\AE}RTa, a novel transformer-based deep neural ensemble architecture for
misinformation detection. MisRoB{\AE}RTa takes advantage of two transformers
(BART \& RoBERTa) to improve the classification performance. We also
benchmarked and evaluated the performances of multiple transformers on the task
of misinformation detection. For training and testing, we used a large
real-world news articles dataset labeled with 10 classes, addressing two
shortcomings in the current research: increasing the size of the dataset from
small to large, and moving the focus of fake news detection from binary
classification to multi-class classification. For this dataset, we manually
verified the content of the news articles to ensure that they were correctly
labeled. The experimental results show that the accuracy of transformers on the
misinformation detection problem was significantly influenced by the method
employed to learn the context, dataset size, and vocabulary dimension. We
observe empirically that the best accuracy performance among the classification
models that use only one transformer is obtained by BART, while DistilRoBERTa
obtains the best accuracy in the least amount of time required for fine-tuning
and training. The proposed MisRoB{\AE}RTa outperforms the other transformer
models in the task of misinformation detection. To arrive at this conclusion,
we performed ample ablation and sensitivity testing with MisRoB{\AE}RTa on two
datasets.

本论文提出了一种基于 transformer 的深度神经集成体系结构（MisRoBÆRTa），用于对谣言的检测，基于一个大型真实新闻文章数据集进行了训练和测试，并且使用两个 transformer（BART＆RoBERTa）来提高分级性能，成果显示 MisRoBÆRTa 在谣言检测任务上表现优于其他 transformer 模型。