We present Subtractive Training, a simple and novel method for synthesizing
individual musical instrument stems given other instruments as context. This
method pairs a dataset of complete music mixes with 1) a variant of the dataset
lacking a specific stem, and 2) LLM-generated instructions describing how the
missing stem should be reintroduced. We then fine-tune a pretrained
text-to-audio diffusion model to generate the missing instrument stem, guided
by both the existing stems and the text instruction. Our results demonstrate
Subtractive Training's efficacy in creating authentic drum stems that
seamlessly blend with the existing tracks. We also show that we can use the
text instruction to control the generation of the inserted stem in terms of
rhythm, dynamics, and genre, allowing us to modify the style of a single
instrument in a full song while keeping the remaining instruments the same.
Lastly, we extend this technique to MIDI formats, successfully generating
compatible bass, drum, and guitar parts for incomplete arrangements.

我们提出了减法训练，一种简单而新颖的方法，用于在给定其他乐器作为上下文的情况下合成个别乐器音轨。该方法将一组完整的音乐混音数据集与缺少特定音轨的数据集变体以及由 LLM 生成的描述如何重新引入缺失音轨的文本指令配对。然后，我们对预训练的文本到音频扩散模型进行微调，以根据现有音轨和文本指令生成缺失的乐器音轨。我们的结果表明减法训练在创建与现有音轨无缝融合的真实鼓音轨方面具有有效性。我们还展示了如何使用文本指令控制插入音轨的生成，以在节奏、动态和风格方面修改完整的歌曲中的单个乐器的风格，同时保持其他乐器不变。最后，我们将这种技术扩展到 MIDI 格式，成功为不完整的编曲生成兼容的低音、鼓和吉他部分。