Hateful memes pose a unique challenge for current machine learning systems
because their message is derived from both text- and visual-modalities. To this
effect, Facebook released the Hateful Memes Challenge, a dataset of memes with
pre-extracted text captions, but it is unclear whether these synthetic examples
generalize to `memes in the wild'. In this paper, we collect hateful and
non-hateful memes from Pinterest to evaluate out-of-sample performance on
models pre-trained on the Facebook dataset. We find that memes in the wild
differ in two key aspects: 1) Captions must be extracted via OCR, injecting
noise and diminishing performance of multimodal models, and 2) Memes are more
diverse than `traditional memes', including screenshots of conversations or
text on a plain background. This paper thus serves as a reality check for the
current benchmark of hateful meme detection and its applicability for detecting
real world hate.

本文针对当前机器学习系统对于包含文本和视觉元素的恶意模因的识别能力较弱的问题，研究了使用 OCR 抽取文本、多样化的模因等

野外的 Memes: 评估 Hateful Memes Challenge 数据集的通用性

Memes in the Wild: Assessing the Generalizability of the Hateful Memes  Challenge Dataset

Memes on the Internet are often harmless and sometimes amusing. However, by
using certain types of images, text, or combinations of both, the seemingly
harmless meme becomes a multimodal type of hate speech -- a hateful meme. The
Hateful Memes Challenge is a first-of-its-kind competition which focuses on
detecting hate speech in multimodal memes and it proposes a new data set
containing 10,000+ new examples of multimodal content. We utilize VisualBERT --
which meant to be the BERT of vision and language -- that was trained
multimodally on images and captions and apply Ensemble Learning. Our approach
achieves 0.811 AUROC with an accuracy of 0.765 on the challenge test set and
placed third out of 3,173 participants in the Hateful Memes Challenge.

Hateful Memes Challenge 使用 VisualBERT 多模态训练进行恶意标记检测，在挑战测试集中取得了 0.811 AUROC 和 0.765 的准确率，获得了 3173 名参赛者中的第三名

使用多模态深度学习方法检测恶意表情包：恶意表情包挑战赛中获奖解决方案

Detecting Hate Speech in Memes Using Multimodal Deep Learning  Approaches: Prize-winning solution to Hateful Memes Challenge

This work examines the vulnerability of multimodal (image + text) models to
adversarial threats similar to those discussed in previous literature on
unimodal (image- or text-only) models. We introduce realistic assumptions of
partial model knowledge and access, and discuss how these assumptions differ
from the standard "black-box"/"white-box" dichotomy common in current
literature on adversarial attacks. Working under various levels of these
"gray-box" assumptions, we develop new attack methodologies unique to
multimodal classification and evaluate them on the Hateful Memes Challenge
classification task. We find that attacking multiple modalities yields stronger
attacks than unimodal attacks alone (inducing errors in up to 73% of cases),
and that the unimodal image attacks on multimodal classifiers we explored were
stronger than character-based text augmentation attacks (inducing errors on
average in 45% and 30% of cases, respectively).

本研究针对多模态（图像 + 文本）模型的弱点进行分析，并提出了基于部分模型知识和访问下的攻击方法。同时，我们发现对多模态分类进行的攻击比仅仅对单模态文本或图像分类的攻击更强，而我们尝试的对多模态分类器进行的单模态图像攻击比文本的字符增强攻击更为有效。