Speech accents pose a significant challenge to state-of-the-art automatic
speech recognition (ASR) systems. Degradation in performance across
underrepresented accents is a severe deterrent to the inclusive adoption of
ASR. In this work, we propose a novel accent adaptation approach for end-to-end
ASR systems using cross-attention with a trainable set of codebooks. These
learnable codebooks capture accent-specific information and are integrated
within the ASR encoder layers. The model is trained on accented English speech,
while the test data also contained accents which were not seen during training.
On the Mozilla Common Voice multi-accented dataset, we show that our proposed
approach yields significant performance gains not only on the seen English
accents (up to $37\%$ relative improvement in word error rate) but also on the
unseen accents (up to $5\%$ relative improvement in WER). Further, we
illustrate benefits for a zero-shot transfer setup on the L2Artic dataset. We
also compare the performance with other approaches based on accent adversarial
training.

通过使用可训练的码书的交叉注意力，我们提出了一种针对终端到终端自动语音识别系统的新型口音适应方法，该方法可以捕捉特定口音的信息，并在 ASR 编码器层中进行集成。我们在包含未在训练过程中见过的口音的测试数据上进行训练，结果显示我们的方法不仅在已知的英语口音上获得显著性能提升（词错误率相对改进高达 37%），还在未知口音上获得了最高 5% 的相对改进。同时，我们还在 L2Artic 数据集上展示了零 - shot 传输设置的优势，并与基于口音对抗训练的其他方法进行了性能比较。