Multimodal Sentiment Analysis is an active area of research that leverages
multimodal signals for affective understanding of user-generated videos. The
predominant approach, addressing this task, has been to develop sophisticated
fusion techniques. However, the heterogeneous nature of the signals creates
distributional modality gaps that pose significant challenges. In this paper,
we aim to learn effective modality representations to aid the process of
fusion. We propose a novel framework, MISA, which projects each modality to two
distinct subspaces. The first subspace is modality-invariant, where the
representations across modalities learn their commonalities and reduce the
modality gap. The second subspace is modality-specific, which is private to
each modality and captures their characteristic features. These representations
provide a holistic view of the multimodal data, which is used for fusion that
leads to task predictions. Our experiments on popular sentiment analysis
benchmarks, MOSI and MOSEI, demonstrate significant gains over state-of-the-art
models. We also consider the task of Multimodal Humor Detection and experiment
on the recently proposed UR_FUNNY dataset. Here too, our model fares better
than strong baselines, establishing MISA as a useful multimodal framework.

本文提出了一种基于 MISA 框架的多模态情感分析方法，该方法利用学习到的模态表示来帮助实现多模态信号的融合，从而提高任务预测的准确性。在多个情感分析基准数据集和 UR_FUNNY 数据集上，该方法表现出了明显的优越性，验证了其在多模态分析中的实用性。