Audio editing involves the arbitrary manipulation of audio content through
precise control. Although text-guided diffusion models have made significant
advancements in text-to-audio generation, they still face challenges in finding
a flexible and precise way to modify target events within an audio track. We
present a novel approach, referred to as PPAE, which serves as a general module
for diffusion models and enables precise audio editing. The editing is based on
the input textual prompt only and is entirely training-free. We exploit the
cross-attention maps of diffusion models to facilitate accurate local editing
and employ a hierarchical local-global pipeline to ensure a smoother editing
process. Experimental results highlight the effectiveness of our method in
various editing tasks.

通过输入的文本提示，我们提出了一种名为 PPAE 的新颖方法，它作为扩散模型的通用模块，实现了精确的音频编辑，通过扩散模型的交叉注意力图来促进准确的局部编辑，并采用分层的局部 - 全局流程确保更流畅的编辑过程，实验证明了我们方法在各种编辑任务中的有效性。

基于提示的扩散模型精确音频编辑

Prompt-guided Precise Audio Editing with Diffusion Models

Audio editing is applicable for various purposes, such as adding background
sound effects, replacing a musical instrument, and repairing damaged audio.
Recently, some diffusion-based methods achieved zero-shot audio editing by
using a diffusion and denoising process conditioned on the text description of
the output audio. However, these methods still have some problems: 1) they have
not been trained on editing tasks and cannot ensure good editing effects; 2)
they can erroneously modify audio segments that do not require editing; 3) they
need a complete description of the output audio, which is not always available
or necessary in practical scenarios. In this work, we propose AUDIT, an
instruction-guided audio editing model based on latent diffusion models.
Specifically, AUDIT has three main design features: 1) we construct triplet
training data (instruction, input audio, output audio) for different audio
editing tasks and train a diffusion model using instruction and input (to be
edited) audio as conditions and generating output (edited) audio; 2) it can
automatically learn to only modify segments that need to be edited by comparing
the difference between the input and output audio; 3) it only needs edit
instructions instead of full target audio descriptions as text input. AUDIT
achieves state-of-the-art results in both objective and subjective metrics for
several audio editing tasks (e.g., adding, dropping, replacement, inpainting,
super-resolution). Demo samples are available at this https URL

本研究提出 AUDIT，一种基于潜在扩散模型的指导音频编辑模型，通过三元训练数据（指令、输入音频、输出音频）训练扩散模型，同时利用输入和指令生成输出音频，实现只修改需要编辑的音频，且只需要编辑指令而非完整的目标音频描述，取得了多项音频编辑任务上的最优结果。