Recent deep music generation studies have put much emphasis on long-term
generation with structures. However, we are yet to see high-quality,
well-structured whole-song generation. In this paper, we make the first attempt
to model a full music piece under the realization of compositional hierarchy.
With a focus on symbolic representations of pop songs, we define a hierarchical
language, in which each level of hierarchy focuses on the semantics and context
dependency at a certain music scope. The high-level languages reveal whole-song
form, phrase, and cadence, whereas the low-level languages focus on notes,
chords, and their local patterns. A cascaded diffusion model is trained to
model the hierarchical language, where each level is conditioned on its upper
levels. Experiments and analysis show that our model is capable of generating
full-piece music with recognizable global verse-chorus structure and cadences,
and the music quality is higher than the baselines. Additionally, we show that
the proposed model is controllable in a flexible way. By sampling from the
interpretable hierarchical languages or adjusting pre-trained external
representations, users can control the music flow via various features such as
phrase harmonic structures, rhythmic patterns, and accompaniment texture.

本文提出了基于组成层次结构实现完整音乐作品建模的方法，通过定义一种分层语言对流行歌曲的符号化表示进行建模。通过级联扩散模型训练了分层语言模型，实验证明该模型能够生成具有可识别的全局歌词 - 副歌结构和协调音的完整音乐作品，且音乐质量高于基准模型。此外，还展示了该模型具有灵活的可控性，用户可以通过从可解释的分层语言中采样或调整预训练的外部表示来控制音乐的流动。

基于级联扩散模型的符号音乐整曲层次生成

Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded  Diffusion Models

We introduce a Cascaded Diffusion Model (Cas-DM) that improves a Denoising
Diffusion Probabilistic Model (DDPM) by effectively incorporating additional
metric functions in training. Metric functions such as the LPIPS loss have been
proven highly effective in consistency models derived from the score matching.
However, for the diffusion counterparts, the methodology and efficacy of adding
extra metric functions remain unclear. One major challenge is the mismatch
between the noise predicted by a DDPM at each step and the desired clean image
that the metric function works well on. To address this problem, we propose
Cas-DM, a network architecture that cascades two network modules to effectively
apply metric functions to the diffusion model training. The first module,
similar to a standard DDPM, learns to predict the added noise and is unaffected
by the metric function. The second cascaded module learns to predict the clean
image, thereby facilitating the metric function computation. Experiment results
show that the proposed diffusion model backbone enables the effective use of
the LPIPS loss, leading to state-of-the-art image quality (FID, sFID, IS) on
various established benchmarks.

我们引入了一种级联扩散模型（Cas-DM），通过在训练中有效地结合额外的度量函数来改进去噪扩散概率模型（DDPM）。实验结果表明，该扩散模型的骨干部分能够有效地利用 LPIPS 损失函数，在各种已建立的基准测试中实现了最先进的图像质量（FID，sFID，IS）。