Recent advancements in hate speech detection (HSD) in Vietnamese have made
significant progress, primarily attributed to the emergence of
transformer-based pre-trained language models, particularly those built on the
BERT architecture. However, the necessity for specialized fine-tuned models has
resulted in the complexity and fragmentation of developing a multitasking HSD
system. Moreover, most current methodologies focus on fine-tuning general
pre-trained models, primarily trained on formal textual datasets like
Wikipedia, which may not accurately capture human behavior on online platforms.
In this research, we introduce ViHateT5, a T5-based model pre-trained on our
proposed large-scale domain-specific dataset named VOZ-HSD. By harnessing the
power of a text-to-text architecture, ViHateT5 can tackle multiple tasks using
a unified model and achieve state-of-the-art performance across all standard
HSD benchmarks in Vietnamese. Our experiments also underscore the significance
of label distribution in pre-training data on model efficacy. We provide our
experimental materials for research purposes, including the VOZ-HSD dataset,
pre-trained checkpoint, the unified HSD-multitask ViHateT5 model, and related
source code on GitHub publicly.

我们引入了基于 T5 架构的 ViHateT5 模型，该模型在我们提出的大规模领域特定数据集 VOZ-HSD 上进行了预训练，通过利用文本到文本架构的优势，ViHateT5 可以使用统一的模型处理多个任务，并在越南语的所有标准 HSD 基准测试中实现最先进的性能。