While learned video codecs have demonstrated great promise, they have yet to
achieve sufficient efficiency for practical deployment. In this work, we
propose several novel ideas for learned video compression which allow for
improved performance for the low-latency mode (I- and P-frames only) along with
a considerable increase in computational efficiency. In this setting, for
natural videos our approach compares favorably across the entire R-D curve
under metrics PSNR, MS-SSIM and VMAF against all mainstream video standards
(H.264, H.265, AV1) and all ML codecs. At the same time, our approach runs at
least 5x faster and has fewer parameters than all ML codecs which report these
figures.
Our contributions include a flexible-rate framework allowing a single model
to cover a large and dense range of bitrates, at a negligible increase in
computation and parameter count; an efficient backbone optimized for ML-based
codecs; and a novel in-loop flow prediction scheme which leverages prior
information towards more efficient compression.
We benchmark our method, which we call ELF-VC (Efficient, Learned and
Flexible Video Coding) on popular video test sets UVG and MCL-JCV under metrics
PSNR, MS-SSIM and VMAF. For example, on UVG under PSNR, it reduces the BD-rate
by 44% against H.264, 26% against H.265, 15% against AV1, and 35% against the
current best ML codec. At the same time, on an NVIDIA Titan V GPU our approach
encodes/decodes VGA at 49/91 FPS, HD 720 at 19/35 FPS, and HD 1080 at 10/18
FPS.

通过提出几种新颖的方法，如专门用于机器学习的高效骨干网络和基于优化的流预测方案，我们的方法 ELF-VC 在视频压缩的低延迟模式下（只有 I 帧和 P 帧）具有更高的性能和更好的效率，与主流视频标准（如 H.264 、 H.265 、 AV1）和所有机器学习编解码器相比，在比特率范围内具有很好的 PSNR 、 MS-SSIM 、 VMAF 等指标，而且至少比所有机器学习编解码器都快 5 倍，并有更少的参数。