Though Multimodal Sentiment Analysis (MSA) proves effective by utilizing rich
information from multiple sources (e.g., language, video, and audio), the
potential sentiment-irrelevant and conflicting information across modalities
may hinder the performance from being further improved. To alleviate this, we
present Adaptive Language-guided Multimodal Transformer (ALMT), which
incorporates an Adaptive Hyper-modality Learning (AHL) module to learn an
irrelevance/conflict-suppressing representation from visual and audio features
under the guidance of language features at different scales. With the obtained
hyper-modality representation, the model can obtain a complementary and joint
representation through multimodal fusion for effective MSA. In practice, ALMT
achieves state-of-the-art performance on several popular datasets (e.g., MOSI,
MOSEI and CH-SIMS) and an abundance of ablation demonstrates the validity and
necessity of our irrelevance/conflict suppression mechanism.

通过利用多个信息源（如语言、视频和音频），多模态情感分析（MSA）证明了其有效性，但不同模态间可能存在情感不相关和冲突的信息，从而限制了进一步提高性能。为了缓解这个问题，我们提出了自适应语言引导的多模态 Transformer（ALMT），它通过自适应的超模态学习（AHL）模块，在不同尺度上利用语言特征的指导从视觉和音频特征中学习抑制不相关和冲突的表示。通过获取超模态表示，该模型可以通过多模态融合获得互补和联合表示，以实现有效的 MSA。在实践中，ALMT 在几个知名数据集（如 MOSI，MOSEI 和 CH-SIMS）上实现了最先进的性能，并通过大量割舍实验证明了我们的抑制不相关和冲突的机制的有效性和必要性。