Hate speech is a societal problem that has significantly grown through the
Internet. New forms of digital content such as image memes have given rise to
spread of hate using multimodal means, being far more difficult to analyse and
detect compared to the unimodal case. Accurate automatic processing, analysis
and understanding of this kind of content will facilitate the endeavor of
hindering hate speech proliferation through the digital world. To this end, we
propose MemeFier, a deep learning-based architecture for fine-grained
classification of Internet image memes, utilizing a dual-stage modality fusion
module. The first fusion stage produces feature vectors containing modality
alignment information that captures non-trivial connections between the text
and image of a meme. The second fusion stage leverages the power of a
Transformer encoder to learn inter-modality correlations at the token level and
yield an informative representation. Additionally, we consider external
knowledge as an additional input, and background image caption supervision as a
regularizing component. Extensive experiments on three widely adopted
benchmarks, i.e., Facebook Hateful Memes, Memotion7k and MultiOFF, indicate
that our approach competes and in some cases surpasses state-of-the-art. Our
code is available on this https URL

本论文提出了一种基于深度学习的、能够精细地分类互联网图像模因的架构 MemeFier，并在 Facebook Hateful Memes、Memotion7k 和 MultiOFF 等三个广泛采用的基准测试中进行了大量实验，结果表明该方法达到并在某些情况下超过了最先进水平。

MemeFier：图像模因分类的双阶段模态融合

MemeFier: Dual-stage Modality Fusion for Image Meme Classification

Despite the increasingly important role played by image memes, we do not yet
have a solid understanding of the elements that might make a meme go viral on
social media. In this paper, we investigate what visual elements distinguish
image memes that are highly viral on social media from those that do not get
re-shared, across three dimensions: composition, subjects, and target audience.
Drawing from research in art theory, psychology, marketing, and neuroscience,
we develop a codebook to characterize image memes, and use it to annotate a set
of 100 image memes collected from 4chan's Politically Incorrect Board (/pol/).
On the one hand, we find that highly viral memes are more likely to use a
close-up scale, contain characters, and include positive or negative emotions.
On the other hand, image memes that do not present a clear subject the viewer
can focus attention on, or that include long text are not likely to be
re-shared by users.
We train machine learning models to distinguish between image memes that are
likely to go viral and those that are unlikely to be re-shared, obtaining an
AUC of 0.866 on our dataset. We also show that the indicators of virality
identified by our model can help characterize the most viral memes posted on
mainstream online social networks too, as our classifiers are able to predict
19 out of the 20 most popular image memes posted on Twitter and Reddit between
2016 and 2018. Overall, our analysis sheds light on what indicators
characterize viral and non-viral visual content online, and set the basis for
developing better techniques to create or moderate content that is more likely
to catch the viewer's attention.

探讨图片模因产生社交媒体传播热度的视觉元素，根据艺术、心理、营销和神经科学研究，开发了特征码书以注释一组从 4chan 收集的 100 个图像模因，训练机器学习模型以区分可能会病毒性传播的图片模因，结果显示我们的模型可以帮助确定最受欢迎的模因。