Linking sheet music images to audio recordings remains a key problem for the
development of efficient cross-modal music retrieval systems. One of the
fundamental approaches toward this task is to learn a cross-modal embedding
space via deep neural networks that is able to connect short snippets of audio
and sheet music. However, the scarcity of annotated data from real musical
content affects the capability of such methods to generalize to real retrieval
scenarios. In this work, we investigate whether we can mitigate this limitation
with self-supervised contrastive learning, by exposing a network to a large
amount of real music data as a pre-training step, by contrasting randomly
augmented views of snippets of both modalities, namely audio and sheet images.
Through a number of experiments on synthetic and real piano data, we show that
pre-trained models are able to retrieve snippets with better precision in all
scenarios and pre-training configurations. Encouraged by these results, we
employ the snippet embeddings in the higher-level task of cross-modal piece
identification and conduct more experiments on several retrieval
configurations. In this task, we observe that the retrieval quality improves
from 30% up to 100% when real music data is present. We then conclude by
arguing for the potential of self-supervised contrastive learning for
alleviating the annotated data scarcity in multi-modal music retrieval models.

我们通过自监督对比学习的方法，研究了如何减轻多模态音乐检索模型中标注数据稀缺的限制，并提出了经过预训练的模型能够更好地检索音乐片段，并在交叉模态的作品识别任务中获得更好的检索质量的观察结果。