The text-based speech editor allows the editing of speech through intuitive cutting, copying, and pasting operations to speed up the process of editing speech. However, the major drawback of current systems is that edited speech often sounds unnatural due to cut-copy-paste operation. In addition, it is not obvious how to synthesize records according to a new word not appearing in the transcript. This paper proposes a novel end-to-end text-based speech editing method called context-aware mask prediction network (CampNet). The model can simulate the text-based speech editing process by randomly masking part of speech and then predicting the masked region by sensing the speech context. It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript. Secondly, for the possible operation of text-based speech editing, we design three text-based operations based on CampNet: deletion, insertion, and replacement. These operations can cover various situations of speech editing. Thirdly, to synthesize the speech corresponding to long text in insertion and replacement operations, a word-level autoregressive generation method is proposed. Fourthly, we propose a speaker adaptation method using only one sentence for CampNet and explore the ability of few-shot learning based on CampNet, which provides a new idea for speech forgery tasks. The subjective and objective experiments on VCTK and LibriTTS datasets show that the speech editing results based on CampNet are better than TTS technology, manual editing, and VoCo method. We also conduct detailed ablation experiments to explore the effect of the CampNet structure on its performance. Finally, the experiment shows that speaker adaptation with only one sentence can further improve the naturalness of speech. Examples of generated speech can be found at https://hairuo55.github.io/CampNet.

该论文提出了一种新的基于文本的语音编辑方法，称为具有上下文感知的掩蔽预测网络（CampNet），该模型可通过随机屏蔽语音的一部分来模拟文本编辑过程，并通过感知语音上下文预测遮蔽区域。此外，论文还提出了三种基于CampNet的文本编辑操作：删除、插入和替换，以覆盖各种语音编辑场景，并通过使用仅一句话的说话人自适应方法和基于CampNet的少量样本学习探索了语音伪造任务 。经过主观和客观实验，发现基于CampNet的语音编辑结果比TTS技术、手动编辑和VoCo方法更好。

CampNet：基于上下文的端到端文本语音编辑中的掩码预测