Symbolic music understanding, which refers to the understanding of music from
the symbolic data (e.g., MIDI format, but not audio), covers many music
applications such as genre classification, emotion classification, and music
pieces matching. While good music representations are beneficial for these
applications, the lack of training data hinders representation learning.
Inspired by the success of pre-training models in natural language processing,
in this paper, we develop MusicBERT, a large-scale pre-trained model for music
understanding. To this end, we construct a large-scale symbolic music corpus
that contains more than 1 million music songs. Since symbolic music contains
more structural (e.g., bar, position) and diverse information (e.g., tempo,
instrument, and pitch), simply adopting the pre-training techniques from NLP to
symbolic music only brings marginal gains. Therefore, we design several
mechanisms, including OctupleMIDI encoding and bar-level masking strategy, to
enhance pre-training with symbolic music data. Experiments demonstrate the
advantages of MusicBERT on four music understanding tasks, including melody
completion, accompaniment suggestion, genre classification, and style
classification. Ablation studies also verify the effectiveness of our designs
of OctupleMIDI encoding and bar-level masking strategy in MusicBERT.

本文介绍了如何用预训练模型 MusicBERT 进行符号音乐理解，并讨论了其中涉及的关键技术，包括 OctupleMIDI 编码和小节层面的掩蔽策略。实验证明，MusicBERT 在旋律完成、伴奏建议、流派、和风格分类等四项任务中具有优越性。