Non-autoregressive automatic speech recognition (NASR) models have gained
attention due to their parallelism and fast inference. The encoder-based NASR,
e.g. connectionist temporal classification (CTC), can be initialized from the
speech foundation models (SFM) but does not account for any dependencies among
intermediate tokens. The encoder-decoder-based NASR, like CTC alignment-based
single-step non-autoregressive transformer (CASS-NAT), can mitigate the
dependency problem but is not able to efficiently integrate SFM. Inspired by
the success of recent work of speech-text joint pre-training with a shared
transformer encoder, we propose a new encoder-based NASR, UniEnc-CASSNAT, to
combine the advantages of CTC and CASS-NAT. UniEnc-CASSNAT consists of only an
encoder as the major module, which can be the SFM. The encoder plays the role
of both the CASS-NAT encoder and decoder by two forward passes. The first pass
of the encoder accepts the speech signal as input, while the concatenation of
the speech signal and the token-level acoustic embedding is used as the input
for the second pass. Examined on the Librispeech 100h, MyST, and Aishell1
datasets, the proposed UniEnc-CASSNAT achieves state-of-the-art NASR results
and is better or comparable to CASS-NAT with only an encoder and hence, fewer
model parameters. Our codes are publicly available.

提出了一种新的基于编码器的非自回归自动语音识别 (UniEnc-CASSNAT) 模型，将 CTC 和 CASS-NAT 的优势相结合，取得了最先进的非自回归自动语音识别结果，并且比只有编码器的 CASS-NAT 模型具有更少的模型参数。