Transformer models have been introduced into end-to-end speech recognition
with state-of-the-art performance on various tasks owing to their superiority
in modeling long-term dependencies. However, such improvements are usually
obtained through the use of very large neural networks. Transformer models
mainly include two submodules - position-wise feedforward layers and
self-attention (SAN) layers. In this paper, to reduce the model complexity
while maintaining good performance, we propose a simplified self-attention
(SSAN) layer which employs FSMN memory block instead of projection layers to
form query and key vectors for transformer-based end-to-end speech recognition.
We evaluate the SSAN-based and the conventional SAN-based transformers on the
public AISHELL-1, internal 1000-hour and 20,000-hour large-scale Mandarin
tasks. Results show that our proposed SSAN-based transformer model can achieve
over 20% relative reduction in model parameters and 6.7% relative CER reduction
on the AISHELL-1 task. With impressively 20% parameter reduction, our model
shows no loss of recognition performance on the 20,000-hour large-scale task.

本文提出了一种简化的自我注意力（SSAN）层，用于 Transformer 模型的端到端语音识别任务中，以降低模型复杂度和维护良好性能，并在公共 AISHELL-1、内部 1000 小时和 20000 小时大规模普通话任务上评估了 SSAN 基于 Transformer 模型与传统基于自我注意力的模型。结果表明，我们的提出的 SSAN-based transformer 模型在模型参数上可以实现超过 20% 相对减少，在 AISHELL-1 任务上实现了 6.7% 相对语音识别误差率的降低，而且在 20000 小时的大规模任务中，模型失去识别性能。