Online speech recognition, where the model only accesses context to the left,
is an important and challenging use case for ASR systems. In this work, we
investigate augmenting neural encoders for online ASR by incorporating
structured state-space sequence models (S4), which are a family of models that
provide a parameter-efficient way of accessing arbitrarily long left context.
We perform systematic ablation studies to compare variants of S4 models and
propose two novel approaches that combine them with convolutions. We find that
the most effective design is to stack a small S4 using real-valued recurrent
weights with a local convolution, allowing them to work complementarily. Our
best model achieves WERs of 4.01%/8.53% on test sets from Librispeech,
outperforming Conformers with extensively tuned convolution.

在线语音识别中，通过将结构化状态空间序列模型（S4）与卷积相结合，可以提高神经编码器的性能并实现更低的识别错误率。

利用结构化状态空间模型增强在线语音识别的构型

Augmenting conformers with structured state space models for online  speech recognition

Recently, a few novel streaming attention-based sequence-to-sequence (S2S)
models have been proposed to perform online speech recognition with linear-time
decoding complexity. However, in these models, the decisions to generate tokens
are delayed compared to the actual acoustic boundaries since their
unidirectional encoders lack future information. This leads to an inevitable
latency during inference. To alleviate this issue and reduce latency, we
propose several strategies during training by leveraging external hard
alignments extracted from the hybrid model. We investigate to utilize the
alignments in both the encoder and the decoder. On the encoder side, (1)
multi-task learning and (2) pre-training with the framewise classification task
are studied. On the decoder side, we (3) remove inappropriate alignment paths
beyond an acceptable latency during the alignment marginalization, and (4)
directly minimize the differentiable expected latency loss. Experiments on the
Cortana voice search task demonstrate that our proposed methods can
significantly reduce the latency, and even improve the recognition accuracy in
certain cases on the decoder side. We also present some analysis to understand
the behaviors of streaming S2S models.

本文研究在线语音识别中的延迟问题，探究了利用硬对齐来进行多任务学习、预训练、合理删减对齐路径和直接减少预期延迟损失等多项方案，成功地实现了对延迟的缩减，并在某些情况下提高了语音识别准确性。

流式序列到序列语音识别的最小延迟训练策略

Minimum Latency Training Strategies for Streaming Sequence-to-Sequence  ASR

Recently, Transformer has gained success in automatic speech recognition
(ASR) field. However, it is challenging to deploy a Transformer-based
end-to-end (E2E) model for online speech recognition. In this paper, we propose
the Transformer-based online CTC/attention E2E ASR architecture, which contains
the chunk self-attention encoder (chunk-SAE) and the monotonic truncated
attention (MTA) based self-attention decoder (SAD). Firstly, the chunk-SAE
splits the speech into isolated chunks. To reduce the computational cost and
improve the performance, we propose the state reuse chunk-SAE. Sencondly, the
MTA based SAD truncates the speech features monotonically and performs
attention on the truncated features. To support the online recognition, we
integrate the state reuse chunk-SAE and the MTA based SAD into online
CTC/attention architecture. We evaluate the proposed online models on the HKUST
Mandarin ASR benchmark and achieve a 23.66% character error rate (CER) with a
320 ms latency. Our online model yields as little as 0.19% absolute CER
degradation compared with the offline baseline, and achieves significant
improvement over our prior work on Long Short-Term Memory (LSTM) based online
E2E models.

本论文介绍了基于 Transformer 的在线 CTC/Attention E2E ASR 架构，该架构包括块自注意力编码器和基于单调截断注意力的自注意力解码器，通过将块自注意力编码器和基于单调截断注意力的自注意力解码器集成到在线 CTC/Attention 架构中，实现了在线语音识别，与离线基线相比，具有最低为 0.19％的 CER 衰减和显着的性能提升。

基于 Transformer 的 CTC / 注意力机制在线端到端语音识别架构

Transformer-based Online CTC/attention End-to-End Speech Recognition  Architecture

Sequence-to-sequence models with soft attention have been successfully
applied to a wide variety of problems, but their decoding process incurs a
quadratic time and space cost and is inapplicable to real-time sequence
transduction. To address these issues, we propose Monotonic Chunkwise Attention
(MoChA), which adaptively splits the input sequence into small chunks over
which soft attention is computed. We show that models utilizing MoChA can be
trained efficiently with standard backpropagation while allowing online and
linear-time decoding at test time. When applied to online speech recognition,
we obtain state-of-the-art results and match the performance of a model using
an offline soft attention mechanism. In document summarization experiments
where we do not expect monotonic alignments, we show significantly improved
performance compared to a baseline monotonic attention-based model.

本文介绍了一种名为 MoChA 的注意力机制，该机制可以将输入序列自适应地分成小块，以实现在线和线性时间解码，并在在线语音识别和文档摘要等任务中取得了良好的结果。