Recent advances in text-to-music generation models have opened new avenues in
musical creativity. However, music generation usually involves iterative
refinements, and how to edit the generated music remains a significant
challenge. This paper introduces a novel approach to the editing of music
generated by such models, enabling the modification of specific attributes,
such as genre, mood and instrument, while maintaining other aspects unchanged.
Our method transforms text editing to \textit{latent space manipulation} while
adding an extra constraint to enforce consistency. It seamlessly integrates
with existing pretrained text-to-music diffusion models without requiring
additional training. Experimental results demonstrate superior performance over
both zero-shot and certain supervised baselines in style and timbre transfer
evaluations. Additionally, we showcase the practical applicability of our
approach in real-world music editing scenarios.

通过操纵潜空间并添加额外限制，本研究介绍了一种新颖方法来编辑文本生成的音乐，实现修改特定属性（如体裁、情绪和乐器），同时保持其他方面的不变。实验结果表明，在风格和音色转移评估中，与零样本和某些监督基线相比，我们的方法表现出卓越的性能。此外，我们展示了本方法在真实音乐编辑场景中的实际适用性。

MusicMagus：零射手风琴文本音乐编辑通过扩散模型

MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models

Music editing primarily entails the modification of instrument tracks or
remixing in the whole, which offers a novel reinterpretation of the original
piece through a series of operations. These music processing methods hold
immense potential across various applications but demand substantial expertise.
Prior methodologies, although effective for image and audio modifications,
falter when directly applied to music. This is attributed to music's
distinctive data nature, where such methods can inadvertently compromise the
intrinsic harmony and coherence of music. In this paper, we develop InstructME,
an Instruction guided Music Editing and remixing framework based on latent
diffusion models. Our framework fortifies the U-Net with multi-scale
aggregation in order to maintain consistency before and after editing. In
addition, we introduce chord progression matrix as condition information and
incorporate it in the semantic space to improve melodic harmony while editing.
For accommodating extended musical pieces, InstructME employs a chunk
transformer, enabling it to discern long-term temporal dependencies within
music sequences. We tested InstructME in instrument-editing, remixing, and
multi-round editing. Both subjective and objective evaluations indicate that
our proposed method significantly surpasses preceding systems in music quality,
text relevance and harmony. Demo samples are available at
this https URL

我们开发了 InstructME，这是一个基于潜在扩散模型的指导式音乐编辑和混音框架，通过多尺度聚合巩固了 U-Net，并引入和语义空间的和弦进行矩阵作为条件信息以提高旋律和谐。我们的方法在音乐质量、文本相关性和和谐度方面明显优于现有系统。

InstructME: 指令引导的音乐编辑和混音框架，其中包括潜在扩散模型

InstructME: An Instruction Guided Music Edit And Remix Framework with  Latent Diffusion Models

In this paper, we introduce Foley Music, a system that can synthesize
plausible music for a silent video clip about people playing musical
instruments. We first identify two key intermediate representations for a
successful video to music generator: body keypoints from videos and MIDI events
from audio recordings. We then formulate music generation from videos as a
motion-to-MIDI translation problem. We present a Graph$-$Transformer framework
that can accurately predict MIDI event sequences in accordance with the body
movements. The MIDI event can then be converted to realistic music using an
off-the-shelf music synthesizer tool. We demonstrate the effectiveness of our
models on videos containing a variety of music performances. Experimental
results show that our model outperforms several existing systems in generating
music that is pleasant to listen to. More importantly, the MIDI representations
are fully interpretable and transparent, thus enabling us to perform music
editing flexibly. We encourage the readers to watch the demo video with audio
turned on to experience the results.

本文介绍了 Foley Music 系统，该系统能够为视频生成逼真的音乐。该系统将视频转换为基于运动的 MIDI 事件，然后利用 Graph-Transformer 框架精确地预测 MIDI 事件序列并转换为逼真的音乐，同时还具有完全可解释和透明的 MIDI 表示形式，可以实现灵活的音乐编辑。实验结果表明，该系统在生成宜人的音乐方面优于现有的几个系统。