A popular approach to streaming speech translation is to employ a single offline model with a \textit{wait-$k$} policy to support different latency requirements, which is simpler than training multiple online models with different latency constraints. However, there is a mismatch problem in using a model trained with complete utterances for streaming inference with partial input. We demonstrate that speech representations extracted at the end of a streaming input are significantly different from those extracted from a complete utterance. To address this issue, we propose a new approach called Future-Aware Streaming Translation (FAST) that adapts an offline ST model for streaming input. FAST includes a Future-Aware Inference (FAI) strategy that incorporates future context through a trainable masked embedding, and a Future-Aware Distillation (FAD) framework that transfers future context from an approximation of full speech to streaming input. Our experiments on the MuST-C EnDe, EnEs, and EnFr benchmarks show that FAST achieves better trade-offs between translation quality and latency than strong baselines. Extensive analyses suggest that our methods effectively alleviate the aforementioned mismatch problem between offline training and online inference.

我们提出了一种名为FAST的方法，它可以为流输入调整离线ST模型，包括将未来信息集成到推理策略中和将未来信息从全语音的近似传输到流输入的框架，从而解决了离线训练和在线推理之间的不匹配问题，并在MuST-C EnDe，EnEs和EnFr基准测试中比强基线模型实现更好的平衡翻译质量和延迟。

使用未来感知蒸馏和推理对离线语音翻译模型进行流式处理的改进