Recent deep music generation studies have put much emphasis on long-term
generation with structures. However, we are yet to see high-quality,
well-structured whole-song generation. In this paper, we make the first attempt
to model a full music piece under the realization of compositional hierarchy.
With a focus on symbolic representations of pop songs, we define a hierarchical
language, in which each level of hierarchy focuses on the semantics and context
dependency at a certain music scope. The high-level languages reveal whole-song
form, phrase, and cadence, whereas the low-level languages focus on notes,
chords, and their local patterns. A cascaded diffusion model is trained to
model the hierarchical language, where each level is conditioned on its upper
levels. Experiments and analysis show that our model is capable of generating
full-piece music with recognizable global verse-chorus structure and cadences,
and the music quality is higher than the baselines. Additionally, we show that
the proposed model is controllable in a flexible way. By sampling from the
interpretable hierarchical languages or adjusting pre-trained external
representations, users can control the music flow via various features such as
phrase harmonic structures, rhythmic patterns, and accompaniment texture.

本文提出了基于组成层次结构实现完整音乐作品建模的方法，通过定义一种分层语言对流行歌曲的符号化表示进行建模。通过级联扩散模型训练了分层语言模型，实验证明该模型能够生成具有可识别的全局歌词 - 副歌结构和协调音的完整音乐作品，且音乐质量高于基准模型。此外，还展示了该模型具有灵活的可控性，用户可以通过从可解释的分层语言中采样或调整预训练的外部表示来控制音乐的流动。

基于级联扩散模型的符号音乐整曲层次生成

Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded  Diffusion Models

We study the problem of symbolic music generation (e.g., generating piano
rolls), with a technical focus on non-differentiable rule guidance. Musical
rules are often expressed in symbolic form on note characteristics, such as
note density or chord progression, many of which are non-differentiable which
pose a challenge when using them for guided diffusion. We propose Stochastic
Control Guidance (SCG), a novel guidance method that only requires forward
evaluation of rule functions that can work with pre-trained diffusion models in
a plug-and-play way, thus achieving training-free guidance for
non-differentiable rules for the first time. Additionally, we introduce a
latent diffusion architecture for symbolic music generation with high time
resolution, which can be composed with SCG in a plug-and-play fashion. Compared
to standard strong baselines in symbolic music generation, this framework
demonstrates marked advancements in music quality and rule-based
controllability, outperforming current state-of-the-art generators in a variety
of settings. For detailed demonstrations, please visit our project site:
this https URL

我们研究了符号音乐生成中的非可微分规则引导问题，提出了一种名为 Stochastic Control Guidance (SCG) 的新型引导方法，以插拔方式与预训练的扩散模型结合使用，实现了对非可微分规则的无训练引导。我们还引入了具有高时间分辨率的潜在扩散架构，可与 SCG 以插拔方式相结合。与符号音乐生成的标准基线相比，该框架在音乐质量和基于规则的控制性能方面表现出明显的进步，优于当前各种场景下的最先进生成器。

用非可微分规则引导扩散进行符号音乐生成

Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion

In this work, we address the task of video background music generation. Some
previous works achieve effective music generation but are unable to generate
melodious music tailored to a particular video, and none of them considers the
video-music rhythmic consistency. To generate the background music that matches
the given video, we first establish the rhythmic relations between video and
background music. In particular, we connect timing, motion speed, and motion
saliency from video with beat, simu-note density, and simu-note strength from
music, respectively. We then propose CMT, a Controllable Music Transformer that
enables local control of the aforementioned rhythmic features and global
control of the music genre and instruments. Objective and subjective
evaluations show that the generated background music has achieved satisfactory
compatibility with the input videos, and at the same time, impressive music
quality. Code and models are available at
this https URL

本文提出了一种基于 Controllable Music Transformer 的视频背景音乐生成方法，该方法考虑到了视频与背景音乐之间的韵律一致性，可以同时实现对音乐类型、乐器等因素的全局与局部控制，生成的音乐具有较好的兼容性和卓越的质量。