Low-resource accented speech recognition is one of the important challenges
faced by current ASR technology in practical applications. In this study, we
propose a Conformer-based architecture, called Aformer, to leverage both the
acoustic information from large non-accented and limited accented training
data. Specifically, a general encoder and an accent encoder are designed in the
Aformer to extract complementary acoustic information. Moreover, we propose to
train the Aformer in a multi-pass manner, and investigate three
cross-information fusion methods to effectively combine the information from
both general and accent encoders. All experiments are conducted on both the
accented English and Mandarin ASR tasks. Results show that our proposed methods
outperform the strong Conformer baseline by relative 10.2% to 24.5%
word/character error rate reduction on six in-domain and out-of-domain accented
test sets.

本研究提出了一种基于 Conformer 的架构，即 Aformer，利用大型非方言和有限口音训练数据从声学信息中提取互补的音频信息，并提出了多通道训练和三种跨信息融合方法，用于从 Aformer 的通用编码器和口音编码器中有效地融合信息，实验结果表明，该方法在口音英语和普通话 ASR 任务上相对于强基线 Conformer 的 6 个领域内外测试数据的单词 / 字符错误率降低了 10.2％到 24.5％。