We introduce DiffuseST, a low-latency, direct speech-to-speech translation
system capable of preserving the input speaker's voice zero-shot while
translating from multiple source languages into English. We experiment with the
synthesizer component of the architecture, comparing a Tacotron-based
synthesizer to a novel diffusion-based synthesizer. We find the diffusion-based
synthesizer to improve MOS and PESQ audio quality metrics by 23\% each and
speaker similarity by 5\% while maintaining comparable BLEU scores. Despite
having more than double the parameter count, the diffusion synthesizer has
lower latency, allowing the entire model to run more than 5$\times$ faster than
real-time.

DiffuseST 是一种低延迟的直接语音到语音翻译系统，能够在将多种源语言翻译为英语时保持输入讲话人的声音；我们通过比较基于 Tacotron 的合成器和基于扩散的合成器，发现基于扩散的合成器在提高音频质量度量 MOS 和 PESQ 各自达到 23％以及讲话人相似度提高 5％的同时，保持相当的 BLEU 得分；尽管参数数量超过两倍，扩散合成器具有更低的延迟，使整个模型的运行速度超过实时的 5 倍以上。

高效多语言语音转语音翻译的扩散合成器

Diffusion Synthesizer for Efficient Multilingual Speech to Speech  Translation

Detecting out-of-distribution (OOD) inputs is a central challenge for safely
deploying machine learning models in the real world. Existing solutions are
mainly driven by small datasets, with low resolution and very few class labels
(e.g., CIFAR). As a result, OOD detection for large-scale image classification
tasks remains largely unexplored. In this paper, we bridge this critical gap by
proposing a group-based OOD detection framework, along with a novel OOD scoring
function termed MOS. Our key idea is to decompose the large semantic space into
smaller groups with similar concepts, which allows simplifying the decision
boundaries between in- vs. out-of-distribution data for effective OOD
detection. Our method scales substantially better for high-dimensional class
space than previous approaches. We evaluate models trained on ImageNet against
four carefully curated OOD datasets, spanning diverse semantics. MOS
establishes state-of-the-art performance, reducing the average FPR95 by 14.33%
while achieving 6x speedup in inference compared to the previous best method.

本文提出了一种基于组的 OOD 检测框架和一种新颖的 OOD 评分函数 MOS，通过将大语义空间分解成具有相似概念的小组，简化内部与外部数据之间的决策边界，从而使得面对高维度分类空间的场景下，我们的方法比以前的方法更为有效。在四个精心策划的 OOD 数据集上进行了评估，证明了 MOS 的性能优越，相比之前最好的方法，平均 FPR95 降低 14.33％，推理速度提高 6 倍。