Simultaneous translation models play a crucial role in facilitating
communication. However, existing research primarily focuses on text-to-text or
speech-to-text models, necessitating additional cascade components to achieve
speech-to-speech translation. These pipeline methods suffer from error
propagation and accumulate delays in each cascade component, resulting in
reduced synchronization between the speaker and listener. To overcome these
challenges, we propose a novel non-autoregressive generation framework for
simultaneous speech translation (NAST-S2X), which integrates speech-to-text and
speech-to-speech tasks into a unified end-to-end framework. We develop a
non-autoregressive decoder capable of concurrently generating multiple text or
acoustic unit tokens upon receiving fixed-length speech chunks. The decoder can
generate blank or repeated tokens and employ CTC decoding to dynamically adjust
its latency. Experimental results show that NAST-S2X outperforms
state-of-the-art models in both speech-to-text and speech-to-speech tasks. It
achieves high-quality simultaneous interpretation within a delay of less than 3
seconds and provides a 28 times decoding speedup in offline generation.

我们提出了一种新颖的非自回归生成框架用于同时语音翻译 (NAST-S2X)，它将语音到文本和语音到语音任务整合到统一的端到端框架中。实验结果表明，NAST-S2X 在语音到文本和语音到语音任务中优于现有模型，在不到 3 秒的延迟内实现了高质量的同时口译，并在离线生成中提供了 28 倍的解码加速。

一种非自回归生成框架用于端到端同步任意语音翻译

A Non-autoregressive Generation Framework for End-to-End Simultaneous  Speech-to-Any Translation

Event co-occurrences have been proved effective for event extraction (EE) in
previous studies, but have not been considered for event argument extraction
(EAE) recently. In this paper, we try to fill this gap between EE research and
EAE research, by highlighting the question that ``Can EAE models learn better
when being aware of event co-occurrences?''. To answer this question, we
reformulate EAE as a problem of table generation and extend a SOTA prompt-based
EAE model into a non-autoregressive generation framework, called TabEAE, which
is able to extract the arguments of multiple events in parallel. Under this
framework, we experiment with 3 different training-inference schemes on 4
datasets (ACE05, RAMS, WikiEvents and MLEE) and discover that via training the
model to extract all events in parallel, it can better distinguish the semantic
boundary of each event and its ability to extract single event gets
substantially improved. Experimental results show that our method achieves new
state-of-the-art performance on the 4 datasets. Our code is avilable at
this https URL

本文主要介绍了通过利用事件共现特性可以提高事件提取以及事件参数提取质量，提出了一种基于表格生成的非自回归框架 TabEAE，并在 4 个数据集上进行了 3 种不同的训练推理方案实验，结果表明该方法在 4 个数据集上均达到了最优的性能表现。