Non-autoregressive automatic speech recognition (NASR) models have gained
attention due to their parallelism and fast inference. The encoder-based NASR,
e.g. connectionist temporal classification (CTC), can be initialized from the
speech foundation models (SFM) but does not account for any dependencies among
intermediate tokens. The encoder-decoder-based NASR, like CTC alignment-based
single-step non-autoregressive transformer (CASS-NAT), can mitigate the
dependency problem but is not able to efficiently integrate SFM. Inspired by
the success of recent work of speech-text joint pre-training with a shared
transformer encoder, we propose a new encoder-based NASR, UniEnc-CASSNAT, to
combine the advantages of CTC and CASS-NAT. UniEnc-CASSNAT consists of only an
encoder as the major module, which can be the SFM. The encoder plays the role
of both the CASS-NAT encoder and decoder by two forward passes. The first pass
of the encoder accepts the speech signal as input, while the concatenation of
the speech signal and the token-level acoustic embedding is used as the input
for the second pass. Examined on the Librispeech 100h, MyST, and Aishell1
datasets, the proposed UniEnc-CASSNAT achieves state-of-the-art NASR results
and is better or comparable to CASS-NAT with only an encoder and hence, fewer
model parameters. Our codes are publicly available.

提出了一种新的基于编码器的非自回归自动语音识别 (UniEnc-CASSNAT) 模型，将 CTC 和 CASS-NAT 的优势相结合，取得了最先进的非自回归自动语音识别结果，并且比只有编码器的 CASS-NAT 模型具有更少的模型参数。

UniEnc-CASSNAT：一种仅编码器非自回归式语音 SSL 模型的 ASR

UniEnc-CASSNAT: An Encoder-only Non-autoregressive ASR for Speech SSL  Models

This paper works on non-autoregressive automatic speech recognition. A
unimodal aggregation (UMA) is proposed to segment and integrate the feature
frames that belong to the same text token, and thus to learn better feature
representations for text tokens. The frame-wise features and weights are both
derived from an encoder. Then, the feature frames with unimodal weights are
integrated and further processed by a decoder. Connectionist temporal
classification (CTC) loss is applied for training. Compared to the regular CTC,
the proposed method learns better feature representations and shortens the
sequence length, resulting in lower recognition error and computational
complexity. Experiments on three Mandarin datasets show that UMA demonstrates
superior or comparable performance to other advanced non-autoregressive
methods, such as self-conditioned CTC. Moreover, by integrating
self-conditioned CTC into the proposed framework, the performance can be
further noticeably improved.

该论文提出了一种非自回归自动语音识别方法，使用一种单模态聚合（UMA）来分割和整合属于同一文本令牌的特征帧，从而学习更好的文本令牌特征表示。与常规 CTC 相比，该方法学习到更好的特征表示，并缩短了序列长度，从而降低了识别错误和计算复杂度。三个普通话数据集上的实验证明 UMA 在非自回归方法中表现出优越或可比较的性能，例如自条件 CTC。此外，通过将自条件 CTC 集成到该方法中，性能可以进一步显著提高。