Compared to traditional sentiment analysis, which only considers text,
multimodal sentiment analysis needs to consider emotional signals from
multimodal sources simultaneously and is therefore more consistent with the way
how humans process sentiment in real-world scenarios. It involves processing
emotional information from various sources such as natural language, images,
videos, audio, physiological signals, etc. However, although other modalities
also contain diverse emotional cues, natural language usually contains richer
contextual information and therefore always occupies a crucial position in
multimodal sentiment analysis. The emergence of ChatGPT has opened up immense
potential for applying large language models (LLMs) to text-centric multimodal
tasks. However, it is still unclear how existing LLMs can adapt better to
text-centric multimodal sentiment analysis tasks. This survey aims to (1)
present a comprehensive review of recent research in text-centric multimodal
sentiment analysis tasks, (2) examine the potential of LLMs for text-centric
multimodal sentiment analysis, outlining their approaches, advantages, and
limitations, (3) summarize the application scenarios of LLM-based multimodal
sentiment analysis technology, and (4) explore the challenges and potential
research directions for multimodal sentiment analysis in the future.

与传统情感分析相比，多模态情感分析需要同时考虑来自多模态源的情感信号，它更符合人类在现实场景中处理情感的方式。本文综述了近期文本为中心的多模态情感分析任务中的研究，审视了大型语言模型在文本为中心的多模态情感分析中的潜力、方法、优势和局限性，总结了基于大型语言模型的多模态情感分析技术的应用场景，并探讨了未来多模态情感分析的挑战和潜在研究方向。

大型语言模型与以文本为中心的多模态情感分析：调查

Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A  Survey

Facial animation in virtual reality environments is essential for
applications that necessitate clear visibility of the user's face and the
ability to convey emotional signals. In our scenario, we animate the face of an
operator who controls a robotic Avatar system. The use of facial animation is
particularly valuable when the perception of interacting with a specific
individual, rather than just a robot, is intended. Purely keypoint-driven
animation approaches struggle with the complexity of facial movements. We
present a hybrid method that uses both keypoints and direct visual guidance
from a mouth camera. Our method generalizes to unseen operators and requires
only a quick enrolment step with capture of two short videos. Multiple source
images are selected with the intention to cover different facial expressions.
Given a mouth camera frame from the HMD, we dynamically construct the target
keypoints and apply an attention mechanism to determine the importance of each
source image. To resolve keypoint ambiguities and animate a broader range of
mouth expressions, we propose to inject visual mouth camera information into
the latent space. We enable training on large-scale speaking head datasets by
simulating the mouth camera input with its perspective differences and facial
deformations. Our method outperforms a baseline in quality, capability, and
temporal consistency. In addition, we highlight how the facial animation
contributed to our victory at the ANA Avatar XPRIZE Finals.

在虚拟现实环境中，我们提出了一种使用关键点和嘴部摄像头的混合方法来进行面部动画，以传达情感信号和模拟与特定个体互动的感知。我们的方法通过捕获两个短视频进行快速注册，利用多个源图像来覆盖不同的面部表情，并将嘴部摄像头信息注入潜在空间以解决关键点的歧义并实现更广泛范围的嘴部表情动画。该方法在质量、能力和时间一致性方面优于基线，并在 ANA Avatar XPRIZE 决赛中的胜利中突显了面部动画的贡献。