Among the various modes of communication in social media, the use of Internet
memes has emerged as a powerful means to convey political, psychological, and
socio-cultural opinions. Although memes are typically humorous in nature,
recent days have witnessed a proliferation of harmful memes targeted to abuse
various social entities. As most harmful memes are highly satirical and
abstruse without appropriate contexts, off-the-shelf multimodal models may not
be adequate to understand their underlying semantics. In this work, we propose
two novel problem formulations: detecting harmful memes and the social entities
that these harmful memes target. To this end, we present HarMeme, the first
benchmark dataset, containing 3,544 memes related to COVID-19. Each meme went
through a rigorous two-stage annotation process. In the first stage, we labeled
a meme as very harmful, partially harmful, or harmless; in the second stage, we
further annotated the type of target(s) that each harmful meme points to:
individual, organization, community, or society/general public/other. The
evaluation results using ten unimodal and multimodal models highlight the
importance of using multimodal signals for both tasks. We further discuss the
limitations of these models and we argue that more research is needed to
address these problems.

本研究提出了两种新的问题定义：检测有害的互联网模因及其攻击的社会实体，并提供了 HarMeme 数据集。评估结果表明，使用多模态信号对这两个任务都很重要，但是需要进一步研究来解决现有模式的局限性。

检测有害模因及其攻击目标

Detecting Harmful Memes and Their Targets

Internet memes have become powerful means to transmit political,
psychological, and socio-cultural ideas. Although memes are typically humorous,
recent days have witnessed an escalation of harmful memes used for trolling,
cyberbullying, and abuse. Detecting such memes is challenging as they can be
highly satirical and cryptic. Moreover, while previous work has focused on
specific aspects of memes such as hate speech and propaganda, there has been
little work on harm in general. Here, we aim to bridge this gap. We focus on
two tasks: (i)detecting harmful memes, and (ii)identifying the social entities
they target. We further extend a recently released HarMeme dataset, which
covered COVID-19, with additional memes and a new topic: US politics. To solve
these tasks, we propose MOMENTA (MultimOdal framework for detecting harmful
MemEs aNd Their tArgets), a novel multimodal deep neural network that uses
global and local perspectives to detect harmful memes. MOMENTA systematically
analyzes the local and the global perspective of the input meme (in both
modalities) and relates it to the background context. MOMENTA is interpretable
and generalizable, and our experiments show that it outperforms several strong
rivaling approaches.

本研究旨在探索检测和识别有害互联网迷因的方法和框架，对有害网络迷因和它们针对的社会实体进行研究分析，提出了一种名为 MOMENTA 的多模态深度神经网络方法，它可以系统分析输入迷因的局部和全局视角，并将其与背景上下文相关联。实验结果显示，MOMENTA 方法的性能优于其他类似方法。