Currently, the development of Foreign Accent Conversion (FAC) models utilizes
deep neural network architectures, as well as ensembles of neural networks for
speech recognition and speech generation. The use of these models is limited by
architectural features, which does not allow flexible changes in the timbre of
the generated speech and requires the accumulation of context, leading to
increased delays in generation and makes these systems unsuitable for use in
real-time multi-user communication scenarios. We have developed the
non-autoregressive model for real-time accent conversion with voice cloning.
The model generates native-sounding L1 speech with minimal latency based on
input L2 accented speech. The model consists of interconnected modules for
extracting accent, gender, and speaker embeddings, converting speech,
generating spectrograms, and decoding the resulting spectrogram into an audio
signal. The model has the ability to save, clone and change the timbre, gender
and accent of the speaker's voice in real time. The results of the objective
assessment show that the model improves speech quality, leading to enhanced
recognition performance in existing ASR systems. The results of subjective
tests show that the proposed accent and gender encoder improves the generation
quality. The developed model demonstrates high-quality low-latency accent
conversion, voice cloning, and speech enhancement capabilities, making it
suitable for real-time multi-user communication scenarios.

我们开发了一种实时语音转换模型，具备母语感、最小延迟生成和多样性切换音色、性别和语音口音的能力，从而提高语音质量，增强现有 ASR 系统的识别性能，并适用于实时多用户通信场景。

非自回归实时音调转换模型与语音克隆

Non-autoregressive real-time Accent Conversion model with voice cloning

Foreign accent conversion (FAC) is a special application of voice conversion
(VC) which aims to convert the accented speech of a non-native speaker to a
native-sounding speech with the same speaker identity. FAC is difficult since
the native speech from the desired non-native speaker to be used as the
training target is impossible to collect. In this work, we evaluate three
recently proposed methods for ground-truth-free FAC, where all of them aim to
harness the power of sequence-to-sequence (seq2seq) and non-parallel VC models
to properly convert the accent and control the speaker identity. Our
experimental evaluation results show that no single method was significantly
better than the others in all evaluation axes, which is in contrast to
conclusions drawn in previous studies. We also explain the effectiveness of
these methods with the training input and output of the seq2seq model and
examine the design choice of the non-parallel VC model, and show that
intelligibility measures such as word error rates do not correlate well with
subjective accentedness. Finally, our implementation is open-sourced to promote
reproducible research and help future researchers improve upon the compared
systems.

外语口音转换是声音转换的一个特殊应用，通过使用序列到序列模型和非平行语音转换模型，本研究评估了三种无需真实参考数据的外语口音转换方法，发现在所有评估指标上，没有一种方法明显优于其他方法。