Continuous dimensional emotion prediction is a challenging task where the
fusion of various modalities usually achieves state-of-the-art performance such
as early fusion or late fusion. In this paper, we propose a novel multi-modal
fusion strategy named conditional attention fusion, which can dynamically pay
attention to different modalities at each time step. Long-short term memory
recurrent neural networks (LSTM-RNN) is applied as the basic uni-modality model
to capture long time dependencies. The weights assigned to different modalities
are automatically decided by the current input features and recent history
information rather than being fixed at any kinds of situation. Our experimental
results on a benchmark dataset AVEC2015 show the effectiveness of our method
which outperforms several common fusion strategies for valence prediction.

该研究提出了一种名为条件注意融合的新型多模态融合策略，将长短期记忆循环神经网络应用为基础单模态模型，自动决定分配给不同情感模态的权重，获得了在情感价值预测方面优于多种常见融合策略的实验结果。