Multimodal learning has exhibited a significant advantage in affective
analysis tasks owing to the comprehensive information of various modalities,
particularly the complementary information. Thus, many emerging studies focus
on disentangling the modality-invariant and modality-specific representations
from input data and then fusing them for prediction. However, our study shows
that modality-specific representations may contain information that is
irrelevant or conflicting with the tasks, which downgrades the effectiveness of
learned multimodal representations. We revisit the disentanglement issue, and
propose a novel triple disentanglement approach, TriDiRA, which disentangles
the modality-invariant, effective modality-specific and ineffective
modality-specific representations from input data. By fusing only the
modality-invariant and effective modality-specific representations, TriDiRA can
significantly alleviate the impact of irrelevant and conflicting information
across modalities during model training. Extensive experiments conducted on
four benchmark datasets demonstrate the effectiveness and generalization of our
triple disentanglement, which outperforms SOTA methods.

通过三重解缠绑方法，TriDiRA，从输入数据中解释了模态不变、有效模态特定和无效模态特定的表示，并且通过融合仅模态不变和有效模态特定的表示，可以显著减少模态之间无关和冲突信息对模型训练的影响。在四个基准数据集上进行的大量实验证明了我们三重解缠绑方法的效果和泛化性能优于现有技术。