Lyric interpretations can help people understand songs and their lyrics
quickly, and can also make it easier to manage, retrieve and discover songs
efficiently from the growing mass of music archives. In this paper we propose
BART-fusion, a novel model for generating lyric interpretations from lyrics and
music audio that combines a large-scale pre-trained language model with an
audio encoder. We employ a cross-modal attention module to incorporate the
audio representation into the lyrics representation to help the pre-trained
language model understand the song from an audio perspective, while preserving
the language model's original generative performance. We also release the Song
Interpretation Dataset, a new large-scale dataset for training and evaluating
our model. Experimental results show that the additional audio information
helps our model to understand words and music better, and to generate precise
and fluent interpretations. An additional experiment on cross-modal music
retrieval shows that interpretations generated by BART-fusion can also help
people retrieve music more accurately than with the original BART.

本文提出了 BART-fusion 模型，用于从音乐音频和歌词中生成歌词解释，该模型结合了一个大规模预训练语言模型和一个音频编码器。实验结果显示，该模型能够更好地理解歌词和音乐，生成精确流畅的解释，并且可以帮助人们更准确地检索音乐。