We present Mask CTC, a novel non-autoregressive end-to-end automatic speech
recognition (ASR) framework, which generates a sequence by refining outputs of
the connectionist temporal classification (CTC). Neural sequence-to-sequence
models are usually \textit{autoregressive}: each output token is generated by
conditioning on previously generated tokens, at the cost of requiring as many
iterations as the output length. On the other hand, non-autoregressive models
can simultaneously generate tokens within a constant number of iterations,
which results in significant inference time reduction and better suits
end-to-end ASR model for real-world scenarios. In this work, Mask CTC model is
trained using a Transformer encoder-decoder with joint training of mask
prediction and CTC. During inference, the target sequence is initialized with
the greedy CTC outputs and low-confidence tokens are masked based on the CTC
probabilities. Based on the conditional dependence between output tokens, these
masked low-confidence tokens are then predicted conditioning on the
high-confidence tokens. Experimental results on different speech recognition
tasks show that Mask CTC outperforms the standard CTC model (e.g., 17.9% ->
12.1% WER on WSJ) and approaches the autoregressive model, requiring much less
inference time using CPUs (0.07 RTF in Python implementation). All of our codes
will be publicly available.

本文提出了 Mask CTC 框架，它是一种新颖的非自回归端到端自动语音识别（ASR）框架，它通过加强 CTC 的输出来生成序列，并使用 Transformer 编码器 - 解码器进行训练。实验结果表明，相比标准的 CTC 模型，Mask CTC 模型在不同的语音识别任务中表现更优，并显著减少了推理时间。