For Music Information Retrieval downstream tasks, the most common audio
representation is time-frequency-based, such as Mel spectrograms. In order to
identify musical genres, this study explores the possibilities of a new form of
audio representation one of the most usual MIR downstream tasks. Therefore, to
discretely encoding music using deep vector quantization; a novel audio
representation was created for the innovative generative music model i.e.
Jukebox. The effectiveness of Jukebox's audio representation is compared to Mel
spectrograms using a dataset that is almost equivalent to State-of-the-Art
(SOTA) and an almost same transformer design. The results of this study imply
that, at least when the transformers are pretrained using a very modest dataset
of 20k tracks, Jukebox's audio representation is not superior to Mel
spectrograms. This could be explained by the fact that Jukebox's audio
representation does not sufficiently take into account the peculiarities of
human hearing perception. On the other hand, Mel spectrograms are specifically
created with the human auditory sense in mind.

通过将音乐进行深度向量量化，为创新的生成音乐模型 Jukebox 创建了一种新型的音频表示方法，并使用与最先进水平相当的数据集进行比较分析，结果显示 Jukebox 的音频表示方法在预训练使用限定数据集时不如 Mel 频谱图，这可能是因为 Jukebox 的音频表示没有充分考虑到人类听觉感知的特殊性。

音乐信息检索中用于音乐流派识别的一种新型音频表示

A Novel Audio Representation for Music Genre Identification in MIR

Despite the central role that melody plays in music perception, it remains an
open challenge in music information retrieval to reliably detect the notes of
the melody present in an arbitrary music recording. A key challenge in melody
transcription is building methods which can handle broad audio containing any
number of instrument ensembles and musical styles - existing strategies work
well for some melody instruments or styles but not all. To confront this
challenge, we leverage representations from Jukebox (Dhariwal et al. 2020), a
generative model of broad music audio, thereby improving performance on melody
transcription by $20$% relative to conventional spectrogram features. Another
obstacle in melody transcription is a lack of training data - we derive a new
dataset containing $50$ hours of melody transcriptions from crowdsourced
annotations of broad music. The combination of generative pre-training and a
new dataset for this task results in $77$% stronger performance on melody
transcription relative to the strongest available baseline. By pairing our new
melody transcription approach with solutions for beat detection, key
estimation, and chord recognition, we build Sheet Sage, a system capable of
transcribing human-readable lead sheets directly from music audio.
Audio examples can be found at this https URL and code at
this https URL .

利用 Jukebox 描述音频数据，并结合新数据集进行模型训练，在旋律转录方面取得了令人瞩目的进展，从而实现了直接从音频转录可阅读的乐谱的 Sheet Sage 系统。

基于生成式预训练的旋律转录

Melody transcription via generative pre-training

We demonstrate that language models pre-trained on codified
(discretely-encoded) music audio learn representations that are useful for
downstream MIR tasks. Specifically, we explore representations from Jukebox
(Dhariwal et al. 2020): a music generation system containing a language model
trained on codified audio from 1M songs. To determine if Jukebox's
representations contain useful information for MIR, we use them as input
features to train shallow models on several MIR tasks. Relative to
representations from conventional MIR models which are pre-trained on tagging,
we find that using representations from Jukebox as input features yields 30%
stronger performance on average across four MIR tasks: tagging, genre
classification, emotion recognition, and key detection. For key detection, we
observe that representations from Jukebox are considerably stronger than those
from models pre-trained on tagging, suggesting that pre-training via codified
audio language modeling may address blind spots in conventional approaches. We
interpret the strength of Jukebox's representations as evidence that modeling
audio instead of tags provides richer representations for MIR.

采用编码音频预训练的语言模型，可以为音乐信息检索任务提供更有用的特征，相较于以标签为预训练的传统模型，使用编码音频预训练模型可以提供更强的检索性能，并能够弥补传统方法中的盲点。

基于编码的音频语言建模为音乐信息检索学习有用表征

Codified audio language modeling learns useful representations for music  information retrieval

We introduce Jukebox, a model that generates music with singing in the raw
audio domain. We tackle the long context of raw audio using a multi-scale
VQ-VAE to compress it to discrete codes, and modeling those using
autoregressive Transformers. We show that the combined model at scale can
generate high-fidelity and diverse songs with coherence up to multiple minutes.
We can condition on artist and genre to steer the musical and vocal style, and
on unaligned lyrics to make the singing more controllable. We are releasing
thousands of non cherry-picked samples at this https URL, along
with model weights and code at this https URL

Jukebox 是一款基于 VQ-VAE 和 Transformer 的原始音频领域音乐生成模型，可以生成高保真度、多样化且长达数分钟的音乐，在音乐和声音风格上可以受到艺术家和类型的影响，并且根据未对齐的歌词进行调整，其生成音乐样本、模型权重和代码已经发布在指定链接上。