We consider the task of multimodal music mood prediction based on the audio
signal and the lyrics of a track. We reproduce the implementation of
traditional feature engineering based approaches and propose a new model based
on deep learning. We compare the performance of both approaches on a database
containing 18,000 tracks with associated valence and arousal values and show
that our approach outperforms classical models on the arousal detection task,
and that both approaches perform equally on the valence prediction task. We
also compare the a posteriori fusion with fusion of modalities optimized
simultaneously with each unimodal model, and observe a significant improvement
of valence prediction. We release part of our database for comparison purposes.

本研究考虑了基于音频信号和歌词的多模态音乐情绪预测任务，复现传统特征工程方法的基础上提出了一种基于深度学习的新模型，证明了新方法相较于传统模型更擅长于唤起用户的注意，而在情感预测上两种方法表现相当。此外，我们还比较了不同的融合方法，在同时优化各单模态模型时取得了较大的提升，并向公众发布部分数据以供参考。