Social media has created a global network where people can easily access and
exchange vast information. This information gives rise to a variety of
opinions, reflecting both positive and negative viewpoints. GIFs stand out as a
multimedia format offering a visually engaging way for users to communicate. In
this research, we propose a multimodal framework that integrates visual and
textual features to predict the GIF sentiment. It also incorporates attributes
including face emotion detection and OCR generated captions to capture the
semantic aspects of the GIF. The developed classifier achieves an accuracy of
82.7% on Twitter GIFs, which is an improvement over state-of-the-art models.
Moreover, we have based our research on the ReactionGIF dataset, analysing the
variance in sentiment perceived by the author and sentiment induced in the
reader

在这项研究中，我们提出了一个多模态框架，整合了视觉和文字特征来预测 GIF 表情的情感，并结合面部情绪检测和 OCR 生成的字幕来捕捉 GIF 的语义方面。通过在 Twitter GIF 上进行实验，我们的分类器达到了 82.7% 的准确率，优于最先进的模型。此外，我们基于 ReactionGIF 数据集进行研究，分析了作者和读者感知的情感差异。