Automated sentiment analysis using Large Language Model (LLM)-based models
like ChatGPT, Gemini or LLaMA2 is becoming widespread, both in academic
research and in industrial applications. However, assessment and validation of
their performance in case of ambiguous or ironic text is still poor. In this
study, we constructed nuanced and ambiguous scenarios, we translated them in 10
languages, and we predicted their associated sentiment using popular LLMs. The
results are validated against post-hoc human responses. Ambiguous scenarios are
often well-coped by ChatGPT and Gemini, but we recognise significant biases and
inconsistent performance across models and evaluated human languages. This work
provides a standardised methodology for automated sentiment analysis evaluation
and makes a call for action to further improve the algorithms and their
underlying data, to improve their performance, interpretability and
applicability.

使用大型语言模型（LLM）的自动情感分析在学术研究和工业应用中越来越普遍，但在处理模糊或讽刺文本的性能评估和验证方面仍不够完善。本研究构建了细致和模糊的场景，将其翻译成 10 种语言，并使用流行的 LLM 预测其关联的情感。结果经过后续人为响应的验证。ChatGPT 和 Gemini 通常能够很好地处理模糊场景，但我们也发现了在不同模型和评估的人类语言之间存在显著偏见和不一致的性能。本研究提供了自动情感分析评估的标准化方法，并呼吁进一步改进算法和其基础数据，以提高其性能、可解释性和适用性。

ChatGPT 与 Gemini 与 LLaMA 在多语言情感分析中的比较

ChatGPT vs Gemini vs LLaMA on Multilingual Sentiment Analysis

Social media networks have become a significant aspect of people's lives,
serving as a platform for their ideas, opinions and emotions. Consequently,
automated sentiment analysis (SA) is critical for recognising people's feelings
in ways that other information sources cannot. The analysis of these feelings
revealed various applications, including brand evaluations, YouTube film
reviews and healthcare applications. As social media continues to develop,
people post a massive amount of information in different forms, including text,
photos, audio and video. Thus, traditional SA algorithms have become limited,
as they do not consider the expressiveness of other modalities. By including
such characteristics from various material sources, these multimodal data
streams provide new opportunities for optimising the expected results beyond
text-based SA. Our study focuses on the forefront field of multimodal SA, which
examines visual and textual data posted on social media networks. Many people
are more likely to utilise this information to express themselves on these
platforms. To serve as a resource for academics in this rapidly growing field,
we introduce a comprehensive overview of textual and visual SA, including data
pre-processing, feature extraction techniques, sentiment benchmark datasets,
and the efficacy of multiple classification methodologies suited to each field.
We also provide a brief introduction of the most frequently utilised data
fusion strategies and a summary of existing research on visual-textual SA.
Finally, we highlight the most significant challenges and investigate several
important sentiment applications.

本文研究了多模态情感分析领域，探讨了社交媒体网络上发布的视觉和文字数据，并提供了包括数据预处理、特征提取技术、情感基准数据集以及适用于不同领域的多种分类方法和数据融合策略。同时，我们还介绍了该领域面临的主要挑战和关键的情感应用。