Trends and opinion mining in social media increasingly focus on novel
interactions involving visual media, like images and short videos, in addition
to text. In this work, we tackle the problem of visual sentiment analysis of
social media images -- specifically, the prediction of image sentiment
polarity. While previous work relied on manually labeled training sets, we
propose an automated approach for building sentiment polarity classifiers based
on a cross-modal distillation paradigm; starting from scraped multimodal (text
+ images) data, we train a student model on the visual modality based on the
outputs of a textual teacher model that analyses the sentiment of the
corresponding textual modality. We applied our method to randomly collected
images crawled from Twitter over three months and produced, after automatic
cleaning, a weakly-labeled dataset of $\sim$1.5 million images. Despite
exploiting noisy labeled samples, our training pipeline produces classifiers
showing strong generalization capabilities and outperforming the current state
of the art on five manually labeled benchmarks for image sentiment polarity
prediction.

本论文研究了社交媒体中的图像情感分析问题，提出了一种基于跨模态蒸馏范式的自动建立情感极性分类器的方法，即从文本数据和图像数据构建出学生模型，并在 Twitter 上收集的随机抓取的图像数据集上对模型进行了评估，证明了其在情感极性预测方面优于现有的方法。