Simultaneous speech translation (SimulST) is a challenging task that aims to directly translate streaming speech before the complete input is observed. A SimulST system generally includes two important components: the pre-decision that aggregates the speech information, and the policy that decides read or write. While recent works had proposed a variety of strategies to improve the pre-decision, they mostly adopt the fixed wait-k policy. The adaptive policies are rarely explored. We propose to model the adaptive policy using the Continuous Integrate-and-Fire (CIF). In our proposed model, the CIF is not only responsible for aggregating speech information, but also deciding when to read or write. To adapt the CIF to SimulST task, we propose two modifications: a token-level quantity loss or an infinite lookback attention. We show that our model can learn an adaptive policy effectively, achieving comparable or superior performance to MMA at lower latency, while being more efficient to train.

本文提出了一种模型, 通过采用连续集成与发射（CIF）方法, 以适应策略来改进预判，从而在实时的语音翻译中取得更好的性能，实验结果表明其优于多头注意力（MMA）方法, 并且具有更好的对长话语的泛化能力。

探索连续积分-火式神经元在自适应同声翻译中的应用