We introduce DiffuseST, a low-latency, direct speech-to-speech translation
system capable of preserving the input speaker's voice zero-shot while
translating from multiple source languages into English. We experiment with the
synthesizer component of the architecture, comparing a Tacotron-based
synthesizer to a novel diffusion-based synthesizer. We find the diffusion-based
synthesizer to improve MOS and PESQ audio quality metrics by 23\% each and
speaker similarity by 5\% while maintaining comparable BLEU scores. Despite
having more than double the parameter count, the diffusion synthesizer has
lower latency, allowing the entire model to run more than 5$\times$ faster than
real-time.

DiffuseST 是一种低延迟的直接语音到语音翻译系统，能够在将多种源语言翻译为英语时保持输入讲话人的声音；我们通过比较基于 Tacotron 的合成器和基于扩散的合成器，发现基于扩散的合成器在提高音频质量度量 MOS 和 PESQ 各自达到 23％以及讲话人相似度提高 5％的同时，保持相当的 BLEU 得分；尽管参数数量超过两倍，扩散合成器具有更低的延迟，使整个模型的运行速度超过实时的 5 倍以上。