The vast majority of inference time for RNN Transducer (RNN-T) models today
is spent on decoding. Current state-of-the-art RNN-T decoding implementations
leave the GPU idle ~80% of the time. Leveraging a new CUDA 12.4 feature, CUDA
graph conditional nodes, we present an exact GPU-based implementation of greedy
decoding for RNN-T models that eliminates this idle time. Our optimizations
speed up a 1.1 billion parameter RNN-T model end-to-end by a factor of 2.5x.
This technique can applied to the "label looping" alternative greedy decoding
algorithm as well, achieving 1.7x and 1.4x end-to-end speedups when applied to
1.1 billion parameter RNN-T and Token and Duration Transducer models
respectively. This work enables a 1.1 billion parameter RNN-T model to run only
16% slower than a similarly sized CTC model, contradicting the common belief
that RNN-T models are not suitable for high throughput inference. The
implementation is available in NVIDIA NeMo.

使用 CUDA 图形条件节点，优化了 RNN-T 模型的解码过程，将其速度提升 2.5 倍，同时可以应用于其他解码算法获得 1.7 倍至 1.4 倍的速度优化，使得高吞吐量推理中的 RNN-T 模型性能接近 CTC 模型。

RNN-T 语音识别模型在 GPU 上的光速准确贪婪解码

Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models  on GPU

Cross-device federated learning (FL) protects user privacy by collaboratively
training a model on user devices, therefore eliminating the need for
collecting, storing, and manually labeling user data. While important topics
such as the FL training algorithm, non-IID-ness, and Differential Privacy have
been well studied in the literature, this paper focuses on two challenges of
practical importance for improving on-device ASR: the lack of ground-truth
transcriptions and the scarcity of compute resource and network bandwidth on
edge devices. First, we propose a FL system for on-device ASR domain adaptation
with full self-supervision, which uses self-labeling together with data
augmentation and filtering techniques. The system can improve a strong
Emformer-Transducer based ASR model pretrained on out-of-domain data, using
in-domain audio without any ground-truth transcriptions. Second, to reduce the
training cost, we propose a self-restricted RNN Transducer (SR-RNN-T) loss, a
variant of alignment-restricted RNN-T that uses Viterbi alignments from
self-supervision. To further reduce the compute and network cost, we
systematically explore adapting only a subset of weights in the
Emformer-Transducer. Our best training recipe achieves a $12.9\%$ relative WER
reduction over the strong out-of-domain baseline, which equals $70\%$ of the
reduction achievable with full human supervision and centralized training.

该论文介绍了一种用于在设备上进行 ASR 领域自适应的联合学习系统，使用自我标记和数据增强和过滤技术来提高强的 Emformer Transducer 的性能，并减少训练和网络成本。

全自监督 ASR 联邦域适应

Federated Domain Adaptation for ASR with Full Self-Supervision

Popular solutions to Named Entity Recognition (NER) include conditional
random fields, sequence-to-sequence models, or utilizing the question-answering
framework. However, they are not suitable for nested and overlapping spans with
large ontologies and for predicting the position of the entities. To fill this
gap, we introduce a new model for NER task -- an RNN transducer (RNN-T). These
models are trained using paired input and output sequences without explicitly
specifying the alignment between them, similar to other seq-to-seq models.
RNN-T models learn the alignment using a loss function that sums over all
alignments. In NER tasks, however, the alignment between words and target
labels are available from the human annotations. We propose a fixed alignment
RNN-T model that utilizes the given alignment, while preserving the benefits of
RNN-Ts such as modeling output dependencies. As a more general case, we also
propose a constrained alignment model where users can specify a relaxation of
the given input alignment and the model will learn an alignment within the
given constraints. In other words, we propose a family of seq-to-seq models
which can leverage alignments between input and target sequences when
available. Through empirical experiments on a challenging real-world medical
NER task with multiple nested ontologies, we demonstrate that our fixed
alignment model outperforms the standard RNN-T model, improving F1-score from
0.70 to 0.74.

提出一种新的序列到序列模型 -- 一个 RNN transducer (RNN-T) 来解决名词实体识别 (NER) 的任务，利用对齐的人类标注信息作为特征来训练模型，并通过实验表明该方法可以提高 F1-score 的表现。