Social media is awash with hateful content, much of which is often veiled
with linguistic and topical diversity. The benchmark datasets used for hate
speech detection do not account for such divagation as they are predominantly
compiled using hate lexicons. However, capturing hate signals becomes
challenging in neutrally-seeded malicious content. Thus, designing models and
datasets that mimic the real-world variability of hate warrants further
investigation.
To this end, we present GOTHate, a large-scale code-mixed crowdsourced
dataset of around 51k posts for hate speech detection from Twitter. GOTHate is
neutrally seeded, encompassing different languages and topics. We conduct
detailed comparisons of GOTHate with the existing hate speech datasets,
highlighting its novelty. We benchmark it with 10 recent baselines. Our
extensive empirical and benchmarking experiments suggest that GOTHate is hard
to classify in a text-only setup. Thus, we investigate how adding endogenous
signals enhances the hate speech detection task. We augment GOTHate with the
user's timeline information and ego network, bringing the overall data source
closer to the real-world setup for understanding hateful content. Our proposed
solution HEN-mBERT is a modular, multilingual, mixture-of-experts model that
enriches the linguistic subspace with latent endogenous signals from history,
topology, and exemplars. HEN-mBERT transcends the best baseline by 2.5% and 5%
in overall macro-F1 and hate class F1, respectively. Inspired by our
experiments, in partnership with Wipro AI, we are developing a semi-automated
pipeline to detect hateful content as a part of their mission to tackle online
harm.

该研究旨在提出一种名为 GOTHate 的数据集，包含了多语言和多主题，并引入内源信号来增强恶意言论检测任务。作者开发出了一种多语言模型 HEN-mBERT，其通过历史、拓扑和范例等内源信号来改善语义分析。实验结果表明，该模型相较于其他模型提升了 2.5% 的整体宏 F1 值和 5% 的恶意言论 F1 值。

重新审视仇恨言论基准：从数据筛选到系统部署

Revisiting Hate Speech Benchmarks: From Data Curation to System  Deployment

The quality of web sources has been traditionally evaluated using exogenous
signals such as the hyperlink structure of the graph. We propose a new approach
that relies on endogenous signals, namely, the correctness of factual
information provided by the source. A source that has few false facts is
considered to be trustworthy. The facts are automatically extracted from each
source by information extraction methods commonly used to construct knowledge
bases. We propose a way to distinguish errors made in the extraction process
from factual errors in the web source per se, by using joint inference in a
novel multi-layer probabilistic model. We call the trustworthiness score we
computed Knowledge-Based Trust (KBT). On synthetic data, we show that our
method can reliably compute the true trustworthiness levels of the sources. We
then apply it to a database of 2.8B facts extracted from the web, and thereby
estimate the trustworthiness of 119M webpages. Manual evaluation of a subset of
the results confirms the effectiveness of the method.

本研究提出了一种新方法，可以通过利用每个网页提供的事实准确性来评估其可信度，使用多层概率模型实现事实提取中的错误区分。通过对 2.8B 条事实的数据库进行估计，结果证实了该方法的有效性。