There is a rapidly growing need for multimodal content moderation (CM) as
more and more content on social media is multimodal in nature. Existing
unimodal CM systems may fail to catch harmful content that crosses modalities
(e.g., memes or videos), which may lead to severe consequences. In this paper,
we present a novel CM model, Asymmetric Mixed-Modal Moderation (AM3), to target
multimodal and unimodal CM tasks. Specifically, to address the asymmetry in
semantics between vision and language, AM3 has a novel asymmetric fusion
architecture that is designed to not only fuse the common knowledge in both
modalities but also to exploit the unique information in each modality. Unlike
previous works that focus on fusing the two modalities while overlooking the
intrinsic difference between the information conveyed in multimodality and in
unimodality (asymmetry in modalities), we propose a novel cross-modality
contrastive loss to learn the unique knowledge that only appears in
multimodality. This is critical as some harmful intent may only be conveyed
through the intersection of both modalities. With extensive experiments, we
show that AM3 outperforms all existing state-of-the-art methods on both
multimodal and unimodal CM benchmarks.

本文介绍了一种新的混合多模式内容调节模型，AM3，用于针对多模式和单模式内容调节任务。我们提出了一种新的跨模态对比损失方法来学习仅出现在多模态内容中的独特知识，并证明了该方法在多模态和单模态内容调节基准测试中优于所有现有的最先进方法。