This paper presents BERT-CTC, a novel formulation of end-to-end speech
recognition that adapts BERT for connectionist temporal classification (CTC).
Our formulation relaxes the conditional independence assumptions used in
conventional CTC and incorporates linguistic knowledge through the explicit
output dependency obtained by BERT contextual embedding. BERT-CTC attends to
the full contexts of the input and hypothesized output sequences via the
self-attention mechanism. This mechanism encourages a model to learn
inner/inter-dependencies between the audio and token representations while
maintaining CTC's training efficiency. During inference, BERT-CTC combines a
mask-predict algorithm with CTC decoding, which iteratively refines an output
sequence. The experimental results reveal that BERT-CTC improves over
conventional approaches across variations in speaking styles and languages.
Finally, we show that the semantic representations in BERT-CTC are beneficial
towards downstream spoken language understanding tasks.

本文提出了 BERT-CTC，一种新的端到端语音识别形式，采用 BERT 来适应连接时间分类（CTC），通过自注意机制，BERT-CTC 关注输入和输出序列的完整上下文，并在保持 CTC 训练效率的同时学习音频和标记表示之间的内部 / 交叉依赖关系，并通过 CTC 解码将 mask-predict 算法与 CTC 解码相结合以迭代地细化输出序列。实验结果表明 BERT-CTC 在说话风格和语言变化的情况下都优于传统方法，并且 BERT-CTC 中的语义表示对下游口语理解任务有益。