Self-attention is an attention mechanism that learns a representation by
relating different positions in the sequence. The transformer, which is a
sequence model solely based on self-attention, and its variants achieved
state-of-the-art results in many natural language processing tasks. Since music
composes its semantics based on the relations between components in sparse
positions, adopting the self-attention mechanism to solve music information
retrieval (MIR) problems can be beneficial. Hence, we propose a self-attention
based deep sequence model for music tagging. The proposed architecture consists
of shallow convolutional layers followed by stacked Transformer encoders.
Compared to conventional approaches using fully convolutional or recurrent
neural networks, our model is more interpretable while reporting competitive
results. We validate the performance of our model with the MagnaTagATune and
the Million Song Dataset. In addition, we demonstrate the interpretability of
the proposed architecture with a heat map visualization.

提出了一种基于自注意力机制的深度序列模型，用于解决音乐信息检索问题，并且相比于传统的卷积和递归神经网络方法，该模型更具可解释性且具有竞争力的结果。