Simultaneous Speech-to-Text translation serves a critical role in real-time
crosslingual communication. Despite the advancements in recent years,
challenges remain in achieving stability in the translation process, a concern
primarily manifested in the flickering of partial results. In this paper, we
propose a novel revision-controllable method designed to address this issue.
Our method introduces an allowed revision window within the beam search pruning
process to screen out candidate translations likely to cause extensive
revisions, leading to a substantial reduction in flickering and, crucially,
providing the capability to completely eliminate flickering. The experiments
demonstrate the proposed method can significantly improve the decoding
stability without compromising substantially on the translation quality.

我们提出了一种新颖的可修订的方法，通过引入允许修订的窗口来筛选候选翻译，从而解决了实时跨语言沟通中的翻译不稳定问题，并且在不大幅度降低翻译质量的情况下，显著提高了解码稳定性。

提高同时语音翻译的稳定性：一种可控修订解码方法

Improving Stability in Simultaneous Speech Translation: A  Revision-Controllable Decoding Approach

ESPnet-ST-v2 is a revamp of the open-source ESPnet-ST toolkit necessitated by
the broadening interests of the spoken language translation community.
ESPnet-ST-v2 supports 1) offline speech-to-text translation (ST), 2)
simultaneous speech-to-text translation (SST), and 3) offline speech-to-speech
translation (S2ST) -- each task is supported with a wide variety of approaches,
differentiating ESPnet-ST-v2 from other open source spoken language translation
toolkits. This toolkit offers state-of-the-art architectures such as
transducers, hybrid CTC/attention, multi-decoders with searchable
intermediates, time-synchronous blockwise CTC/attention, Translatotron models,
and direct discrete unit models. In this paper, we describe the overall design,
example models for each task, and performance benchmarking behind ESPnet-ST-v2,
which is publicly available at this https URL

ESPnet-ST-v2 是一个开源工具包，它支持离线语音到文本翻译、同声传译以及离线语音到语音翻译等多种任务，提供了包括转录器、混合 CTC/attention、多解码器、时间同步分块 CTC/attention、Translatotron 模型和直接离散单元模型等最先进的架构。

ESPnet-ST-v2: 多用途口语翻译工具包

ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit

Simultaneous speech-to-text translation is widely useful in many scenarios.
The conventional cascaded approach uses a pipeline of streaming ASR followed by
simultaneous MT, but suffers from error propagation and extra latency. To
alleviate these issues, recent efforts attempt to directly translate the source
speech into target text simultaneously, but this is much harder due to the
combination of two separate tasks. We instead propose a new paradigm with the
advantages of both cascaded and end-to-end approaches. The key idea is to use
two separate, but synchronized, decoders on streaming ASR and direct
speech-to-text translation (ST), respectively, and the intermediate results of
ASR guide the decoding policy of (but is not fed as input to) ST. During
training time, we use multitask learning to jointly learn these two tasks with
a shared encoder. En-to-De and En-to-Es experiments on the MuSTC dataset
demonstrate that our proposed technique achieves substantially better
translation quality at similar levels of latency.

本文提出一种新型的语音翻译范例，通过使用两个分离但同步的解码器，一个用于流式 ASR, 一个用于直接语音翻译，并通过 ASR 生成的中间结果指导直接语音翻译，实现了流畅度更高的翻译质量。