Multi-modal deep learning techniques for matching free-form text with music
have shown promising results in the field of Music Information Retrieval (MIR).
Prior work is often based on large proprietary data while publicly available
datasets are few and small in size. In this study, we present WikiMuTe, a new
and open dataset containing rich semantic descriptions of music. The data is
sourced from Wikipedia's rich catalogue of articles covering musical works.
Using a dedicated text-mining pipeline, we extract both long and short-form
descriptions covering a wide range of topics related to music content such as
genre, style, mood, instrumentation, and tempo. To show the use of this data,
we train a model that jointly learns text and audio representations and
performs cross-modal retrieval. The model is evaluated on two tasks: tag-based
music retrieval and music auto-tagging. The results show that while our
approach has state-of-the-art performance on multiple tasks, but still observe
a difference in performance depending on the data used for training.

使用多模态深度学习技术匹配自由形式的文本与音乐在音乐信息检索领域显示出有希望的结果。本研究提出了一个包含音乐丰富语义描述的新开放数据集 WikiMuTe，数据源于维基百科的音乐作品文章目录。使用专门的文本挖掘流程提取覆盖音乐内容各种主题的长短句描述，例如流派、风格、情感、乐器和节奏。展示了利用该数据集训练了一个联合学习文本和音频表示的模型，并进行跨模态检索。该模型在两个任务上进行评估：基于标签的音乐检索和音乐自动标注。结果表明，尽管我们的方法在多个任务上具有最先进的性能，但仍然观察到性能差异取决于用于训练的数据。