Music editing primarily entails the modification of instrument tracks or
remixing in the whole, which offers a novel reinterpretation of the original
piece through a series of operations. These music processing methods hold
immense potential across various applications but demand substantial expertise.
Prior methodologies, although effective for image and audio modifications,
falter when directly applied to music. This is attributed to music's
distinctive data nature, where such methods can inadvertently compromise the
intrinsic harmony and coherence of music. In this paper, we develop InstructME,
an Instruction guided Music Editing and remixing framework based on latent
diffusion models. Our framework fortifies the U-Net with multi-scale
aggregation in order to maintain consistency before and after editing. In
addition, we introduce chord progression matrix as condition information and
incorporate it in the semantic space to improve melodic harmony while editing.
For accommodating extended musical pieces, InstructME employs a chunk
transformer, enabling it to discern long-term temporal dependencies within
music sequences. We tested InstructME in instrument-editing, remixing, and
multi-round editing. Both subjective and objective evaluations indicate that
our proposed method significantly surpasses preceding systems in music quality,
text relevance and harmony. Demo samples are available at
this https URL

我们开发了 InstructME，这是一个基于潜在扩散模型的指导式音乐编辑和混音框架，通过多尺度聚合巩固了 U-Net，并引入和语义空间的和弦进行矩阵作为条件信息以提高旋律和谐。我们的方法在音乐质量、文本相关性和和谐度方面明显优于现有系统。