Multimodal sentiment analysis and depression estimation are two important
research topics that aim to predict human mental states using multimodal data.
Previous research has focused on developing effective fusion strategies for
exchanging and integrating mind-related information from different modalities.
Some MLP-based techniques have recently achieved considerable success in a
variety of computer vision tasks. Inspired by this, we explore multimodal
approaches with a feature-mixing perspective in this study. To this end, we
introduce CubeMLP, a multimodal feature processing framework based entirely on
MLP. CubeMLP consists of three independent MLP units, each of which has two
affine transformations. CubeMLP accepts all relevant modality features as input
and mixes them across three axes. After extracting the characteristics using
CubeMLP, the mixed multimodal features are flattened for task predictions. Our
experiments are conducted on sentiment analysis datasets: CMU-MOSI and
CMU-MOSEI, and depression estimation dataset: AVEC2019. The results show that
CubeMLP can achieve state-of-the-art performance with a much lower computing
cost.

本研究采用 CubeMLP 基于 MLP 的多模态特征处理框架，将不同来源的情感分析和抑郁症评估数据混合并使用，证实其可在计算成本更低的状态下取得最先进的性能。