The presence of abusive content on social media platforms is undesirable as
it severely impedes healthy and safe social media interactions. While automatic
abuse detection has been widely explored in textual domain, audio abuse
detection still remains unexplored. In this paper, we attempt abuse detection
in conversational audio from a multimodal perspective in a multilingual social
media setting. Our key hypothesis is that along with the modelling of audio,
incorporating discriminative information from other modalities can be highly
beneficial for this task. Our proposed method, MADA, explicitly focuses on two
modalities other than the audio itself, namely, the underlying emotions
expressed in the abusive audio and the semantic information encapsulated in the
corresponding textual form. Observations prove that MADA demonstrates gains
over audio-only approaches on the ADIMA dataset. We test the proposed approach
on 10 different languages and observe consistent gains in the range 0.6%-5.2%
by leveraging multiple modalities. We also perform extensive ablation
experiments for studying the contributions of every modality and observe the
best results while leveraging all the modalities together. Additionally, we
perform experiments to empirically confirm that there is a strong correlation
between underlying emotions and abusive behaviour.

本文提出了一种多模态方法（MADA），以从社交媒体上的多语言对话音频中检测滥用内容为目标，并证明利用其他模态的判别信息对音频建模可以极大地提高性能。实验证明，MADA 可以在 ADIMA 数据集上取得比仅使用音频的方法更好的表现，并且在测试了 10 种不同语言后，在多模态结合的情况下可以获得 0.6％-5.2％ 的一致增益。 此外，我们还做了实验证明了潜在情绪和虐待行为之间存在强烈的相关性。