Transformers have been the dominant architecture for Speech Translation in
recent years, achieving significant improvements in translation quality. Since
speech signals are longer than their textual counterparts, and due to the
quadratic complexity of the Transformer, a down-sampling step is essential for
its adoption in Speech Translation. Instead, in this research, we propose to
ease the complexity by using a Perceiver encoder to map the speech inputs to a
fixed-length latent representation. Furthermore, we introduce a novel way of
training Perceivers, with Dynamic Latent Access (DLA), unlocking larger latent
spaces without any additional computational overhead. Speech-to-Text Perceivers
with DLA can match the performance of Transformer baselines across three
language pairs in MuST-C. Finally, a DLA-trained model is easily adaptable to
DLA at inference, and can be flexibly deployed with various computational
budgets, without significant drops in translation quality.

该研究提出了一种使用感知器编码器和动态潜在访问训练的语音到文本转换技术，这种技术在 MuST-C 数据集上的三种语言中可以与 Transformer 基线模型的性能相匹配，并且可以在推理过程中灵活部署，适应多种计算预算，而不会导致显著降低翻译质量。