Neural video codecs have recently become competitive with standard codecs
such as HEVC in the low-delay setting. However, most neural codecs are large
floating-point networks that use pixel-dense warping operations for temporal
modeling, making them too computationally expensive for deployment on mobile
devices. Recent work has demonstrated that running a neural decoder in real
time on mobile is feasible, but shows this only for 720p RGB video, while the
YUV420 format is more commonly used in production. This work presents the first
neural video codec that decodes 1080p YUV420 video in real time on a mobile
device. Our codec relies on two major contributions. First, we design an
efficient codec that uses a block-based motion compensation algorithm available
on the warping core of the mobile accelerator, and we show how to quantize this
model to integer precision. Second, we implement a fast decoder pipeline that
concurrently runs neural network components on the neural signal processor,
parallel entropy coding on the mobile GPU, and warping on the warping core. Our
codec outperforms the previous on-device codec by a large margin with up to 48
% BD-rate savings, while reducing the MAC count on the receiver side by 10x. We
perform a careful ablation to demonstrate the effect of the introduced motion
compensation scheme, and ablate the effect of model quantization.

该研究提出了第一个在移动设备上实时解码 1080p YUV420 视频的神经视频编解码器，它利用基于块的运动补偿算法实现了高效编解码，并通过在神经信号处理器上同时运行神经网络组件、在移动 GPU 上运行并行熵编码以及在加速器的变换核上实现变换，从而在性能上显著超过之前的设备上编解码器。

MobileNVC: 移动设备上实时 1080p 神经视频压缩

MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Device

End-to-end (E2E) models, which directly predict output character sequences
given input speech, are good candidates for on-device speech recognition. E2E
models, however, present numerous challenges: In order to be truly useful, such
models must decode speech utterances in a streaming fashion, in real time; they
must be robust to the long tail of use cases; they must be able to leverage
user-specific context (e.g., contact lists); and above all, they must be
extremely accurate. In this work, we describe our efforts at building an E2E
speech recognizer using a recurrent neural network transducer. In experimental
evaluations, we find that the proposed approach can outperform a conventional
CTC-based model in terms of both latency and accuracy in a number of evaluation
categories.

本文詳細描述了使用遞歸神經網絡轉導器構建端到端語音識別器的努力，該模型可以以流式方式在實時下進行解碼，能夠應對各種需求，能利用特定用戶的上下文信息，並超越基於 CTC 的模型在各評價指標中的性能表現。