Modality representation learning is an important problem for multimodal
sentiment analysis (MSA), since the highly distinguishable representations can
contribute to improving the analysis effect. Previous works of MSA have usually
focused on multimodal fusion strategies, and the deep study of modal
representation learning was given less attention. Recently, contrastive
learning has been confirmed effective at endowing the learned representation
with stronger discriminate ability. Inspired by this, we explore the
improvement approaches of modality representation with contrastive learning in
this study. To this end, we devise a three-stages framework with multi-view
contrastive learning to refine representations for the specific objectives. At
the first stage, for the improvement of unimodal representations, we employ the
supervised contrastive learning to pull samples within the same class together
while the other samples are pushed apart. At the second stage, a
self-supervised contrastive learning is designed for the improvement of the
distilled unimodal representations after cross-modal interaction. At last, we
leverage again the supervised contrastive learning to enhance the fused
multimodal representation. After all the contrast trainings, we next achieve
the classification task based on frozen representations. We conduct experiments
on three open datasets, and results show the advance of our model.

本文探索了应用对比学习改进模态表征的方法，提出了三阶段的多视角对比学习框架，通过监督和自监督对比学习来改进单模态和融合的多模态表征，并成功提高了多模态情感分析任务的效果。