In this paper, we propose an efficient and accurate streaming speech
recognition model based on the FastConformer architecture. We adapted the
FastConformer architecture for streaming applications through: (1) constraining
both the look-ahead and past contexts in the encoder, and (2) introducing an
activation caching mechanism to enable the non-autoregressive encoder to
operate autoregressively during inference. The proposed model is thoughtfully
designed in a way to eliminate the accuracy disparity between the train and
inference time which is common for many streaming models. Furthermore, our
proposed encoder works with various decoder configurations including
Connectionist Temporal Classification (CTC) and RNN-Transducer (RNNT) decoders.
Additionally, we introduced a hybrid CTC/RNNT architecture which utilizes a
shared encoder with both a CTC and RNNT decoder to boost the accuracy and save
computation. We evaluate the proposed model on LibriSpeech dataset and a
multi-domain large scale dataset and demonstrate that it can achieve better
accuracy with lower latency and inference time compared to a conventional
buffered streaming model baseline. We also showed that training a model with
multiple latencies can achieve better accuracy than single latency models while
it enables us to support multiple latencies with a single model. Our
experiments also showed the hybrid architecture would not only speedup the
convergence of the CTC decoder but also improves the accuracy of streaming
models compared to single decoder models.

我们提出了一种基于 FastConformer 架构的高效准确的流式语音识别模型，在编码器中通过约束前向和后向上下文，并引入了激活缓存机制来实现非自回归编码器在推断过程中的自回归操作。我们的模型消除了许多流式模型中常见的训练与推断准确性差异，并能与多种解码器配置一起工作。此外，我们还引入了一种混合 CTC/RNNT 架构，利用共享编码器与 CTC 和 RNNT 解码器结合，以提高准确性并节省计算资源。我们在 LibriSpeech 数据集和多领域大规模数据集上评估了我们的模型，并证明它相对于传统的缓冲流式模型基线具有更高的准确性、更低的延迟和推断时间。同时，我们的实验证明使用多种延迟训练模型可以获得比单一延迟模型更高的准确性，并能够通过一个模型支持多种延迟。我们的实验还表明，相比于单一解码器模型，混合架构不仅加快了 CTC 解码器的收敛速度，而且提高了流式模型的准确性。

基于缓存推断的流式自动语音识别中的有状态快速 Conformer

Stateful FastConformer with Cache-based Inference for Streaming  Automatic Speech Recognition

Unmanned Aerial Vehicles (UAVs) have recently attracted significant attention
due to their outstanding ability to be used in different sectors and serve in
difficult and dangerous areas. Moreover, the advancements in computer vision
and artificial intelligence have increased the use of UAVs in various
applications and solutions, such as forest fires detection and borders
monitoring. However, using deep neural networks (DNNs) with UAVs introduces
several challenges of processing deeper networks and complex models, which
restricts their on-board computation. In this work, we present a strategy
aiming at distributing inference requests to a swarm of resource-constrained
UAVs that classifies captured images on-board and finds the minimum
decision-making latency. We formulate the model as an optimization problem that
minimizes the latency between acquiring images and making the final decisions.
The formulated optimization solution is an NP-hard problem. Hence it is not
adequate for online resource allocation. Therefore, we introduce an online
heuristic solution, namely DistInference, to find the layers placement strategy
that gives the best latency among the available UAVs. The proposed approach is
general enough to be used for different low decision-latency applications as
well as for all CNN types organized into the pipeline of layers (e.g., VGG) or
based on residual blocks (e.g., ResNet).

在本文中，我们提出了一种旨在将推理请求分配到资源受限的 UAV 群体的策略，该策略可在无人机上对捕获的图像进行分类，并找到最小的决策延迟，其模型被制定为一个最小化获取图像与做出最终决策之间延迟的优化问题，我们提出了一种名为 DistInference 的在线启发式解决方案，以找到最佳延迟的分层策略，可以用于不同的低决策延迟应用程序以及所有 CNN 类型。

协同无人机和卷积网络实现高效实时图像识别

Efficient Real-Time Image Recognition Using Collaborative Swarm of UAVs  and Convolutional Networks

Neural architecture search (NAS) approaches aim at automatically finding
novel CNN architectures that fit computational constraints while maintaining a
good performance on the target platform. We introduce a novel efficient
one-shot NAS approach to optimally search for channel numbers, given latency
constraints on a specific hardware. We first show that we can use a black-box
approach to estimate a realistic latency model for a specific inference
platform, without the need for low-level access to the inference computation.
Then, we design a pairwise MRF to score any channel configuration and use
dynamic programming to efficiently decode the best performing configuration,
yielding an optimal solution for the network width search. Finally, we propose
an adaptive channel configuration sampling scheme to gradually specialize the
training phase to the target computational constraints. Experiments on ImageNet
classification show that our approach can find networks fitting the resource
constraints on different target platforms while improving accuracy over the
state-of-the-art efficient networks.

本研究提出了一种高效的一次性神经结构搜索方法，通过优化搜索通道数以求在特定硬件上的延迟限制下，获得较好的神经结构性能，并通过适应性通道策略逐步满足目标计算约束。在 ImageNet 分类的实验中，该方法可以找到符合资源约束的网络解决方案，同时提高了有效网络的准确性，展示出新的颖展性。