The ability to automatically generate music that appropriately matches an
arbitrary input track is a challenging task. We present a novel controllable
system for generating single stems to accompany musical mixes of arbitrary
length. At the core of our method are audio autoencoders that efficiently
compress audio waveform samples into invertible latent representations, and a
conditional latent diffusion model that takes as input the latent encoding of a
mix and generates the latent encoding of a corresponding stem. To provide
control over the timbre of generated samples, we introduce a technique to
ground the latent space to a user-provided reference style during diffusion
sampling. For further improving audio quality, we adapt classifier-free
guidance to avoid distortions at high guidance strengths when generating an
unbounded latent space. We train our model on a dataset of pairs of mixes and
matching bass stems. Quantitative experiments demonstrate that, given an input
mix, the proposed system can generate basslines with user-specified timbres.
Our controllable conditional audio generation framework represents a
significant step forward in creating generative AI tools to assist musicians in
music production.

我们提出了一种新颖的可控制系统，用于生成与任意输入音轨相匹配的单音轨，核心是音频自编码器有效地压缩音频波形样本成可逆的潜在表示，并且条件化的潜在扩散模型以输入音轨的潜在编码生成对应音轨的潜在编码，为了提供对生成样本音色的控制，我们引入了一种在扩散采样期间将潜在空间与用户提供的参考样式进行关联的技术，为了进一步提高音频质量，我们使用无分类器引导的方法避免在生成无界潜在空间时出现失真，我们使用配对的音轨混合组成的数据集对模型进行训练，定量实验证明，给定输入音轨，所提出的系统能够生成用户指定音色的低音线，我们的可控条件音频生成框架在帮助音乐制作方面迈出了重要的一步。

通过潜在扩散实现低音伴奏生成

Bass Accompaniment Generation via Latent Diffusion

The recent success of raw audio waveform synthesis models like WaveNet
motivates a new approach for music synthesis, in which the entire process ---
creating audio samples from a score and instrument information --- is modeled
using generative neural networks. This paper describes a neural music synthesis
model with flexible timbre controls, which consists of a recurrent neural
network conditioned on a learned instrument embedding followed by a WaveNet
vocoder. The learned embedding space successfully captures the diverse
variations in timbres within a large dataset and enables timbre control and
morphing by interpolating between instruments in the embedding space. The
synthesis quality is evaluated both numerically and perceptually, and an
interactive web demo is presented.

本研究提出了一种基于神经网络的音乐合成模型，并使用具有灵活音色控制的嵌入学习来实现乐器的多样性和交互。