Social media platforms are hubs for multimodal information exchange,
encompassing text, images, and videos, making it challenging for machines to
comprehend the information or emotions associated with interactions in online
spaces. Multimodal Large Language Models (MLLMs) have emerged as a promising
solution to address these challenges, yet struggle with accurately interpreting
human emotions and complex contents like misinformation. This paper introduces
MM-Soc, a comprehensive benchmark designed to evaluate MLLMs' understanding of
multimodal social media content. MM-Soc compiles prominent multimodal datasets
and incorporates a novel large-scale YouTube tagging dataset, targeting a range
of tasks from misinformation detection, hate speech detection, and social
context generation. Through our exhaustive evaluation on ten size-variants of
four open-source MLLMs, we have identified significant performance disparities,
highlighting the need for advancements in models' social understanding
capabilities. Our analysis reveals that, in a zero-shot setting, various types
of MLLMs generally exhibit difficulties in handling social media tasks.
However, MLLMs demonstrate performance improvements post fine-tuning,
suggesting potential pathways for improvement.

社交媒体是多模态信息交流的中心，包括文本、图片和视频，这对机器来理解在线空间中的信息或情感相关的交互构成了挑战。本文介绍了 MM-Soc，一个综合性基准，旨在评估多模态大型语言模型对多模态社交媒体内容的理解能力。通过我们对四个开源多模态大型语言模型的十个规模变体的详尽评估，我们发现了重要的性能差异，突出了模型在社交理解能力方面的改进需求。