For the task of speech recognition, the use of more than 30 seconds of
acoustic context during training is uncommon, and under-investigated in
literature. In this work, we examine the effect of scaling the sequence length
used to train/evaluate (dense-attention based) acoustic and language models on
speech recognition performance. For these experiments a dataset of roughly
100,000 pseudo-labelled Spotify podcasts is used, with context lengths of 5
seconds to 1 hour being explored. Zero-shot evaluations on long-format datasets
Earnings-22 and Tedlium demonstrate a benefit from training with around 80
seconds of acoustic context, showing up to a 14.9% relative improvement from a
limited context baseline. Furthermore, we perform a system combination with
long-context transformer language models via beam search for a fully
long-context ASR system, with results that are competitive with the current
state-of-the-art.

使用超过 30 秒的声学上下文来训练语音识别模型在文献中很少见，并且缺乏深入研究。本文通过对超长的声学和语言模型进行训练和评估的序列长度进行了研究，结果表明使用大约 80 秒的声学上下文进行训练可以相对提高 14.9% 的性能，并且与当前最先进的系统相比，通过长上下文转换器语言模型的系统组合可以得到具有竞争力的结果。