Hate speech encompasses verbal, written, or behavioral communication that
targets derogatory or discriminatory language against individuals or groups
based on sensitive characteristics. Automated hate speech detection plays a
crucial role in curbing its propagation, especially across social media
platforms. Various methods, including recent advancements in deep learning,
have been devised to address this challenge. In this study, we introduce
HateTinyLLM, a novel framework based on fine-tuned decoder-only tiny large
language models (tinyLLMs) for efficient hate speech detection. Our
experimental findings demonstrate that the fine-tuned HateTinyLLM outperforms
the pretrained mixtral-7b model by a significant margin. We explored various
tiny LLMs, including PY007/TinyLlama-1.1B-step-50K-105b, Microsoft/phi-2, and
facebook/opt-1.3b, and fine-tuned them using LoRA and adapter methods. Our
observations indicate that all LoRA-based fine-tuned models achieved over 80\%
accuracy.

本研究介绍了一种名为 HateTinyLLM 的基于精调的解码器型微型大型语言模型（tinyLLM）的新框架，用于高效检测仇恨言论。实验证明，精调的 HateTinyLLM 在检测效果上明显优于预训练的 mixtral-7b 模型。我们尝试了多个微型 LLM，包括 PY007/TinyLlama-1.1B-step-50K-105b、Microsoft/phi-2 和 facebook/opt-1.3b，并通过 LoRA 和适配器方法进行了精调。观察结果表明，所有基于 LoRA 的精调模型的准确率都超过 80%。