Emotion semantic inconsistency is an ubiquitous challenge in multi-modal
sentiment analysis (MSA). MSA involves analyzing sentiment expressed across
various modalities like text, audio, and videos. Each modality may convey
distinct aspects of sentiment, due to subtle and nuanced expression of human
beings, leading to inconsistency, which may hinder the prediction of artificial
agents. In this work, we introduce a modality conflicting test set and assess
the performance of both traditional multi-modal sentiment analysis models and
multi-modal large language models (MLLMs). Our findings reveal significant
performance degradation across traditional models when confronted with
semantically conflicting data and point out the drawbacks of MLLMs when
handling multi-modal emotion analysis. Our research presents a new challenge
and offer valuable insights for the future development of sentiment analysis
systems.

多模态情感分析中的情感语义一致性是一个普遍存在的挑战，我们通过引入模态冲突的测试集评估传统的多模态情感分析模型和多模态大型语言模型的性能，发现传统模型在面对语义冲突数据时表现出显著的性能降低，并指出了多模态情感分析中多模态大型语言模型的缺点。我们的研究提出了一个新的挑战，并为情感分析系统的未来发展提供了宝贵的见解。