Conventional video compression approaches use the predictive coding
architecture and encode the corresponding motion information and residual
information. In this paper, taking advantage of both classical architecture in
the conventional video compression method and the powerful non-linear
representation ability of neural networks, we propose the first end-to-end
video compression deep model that jointly optimizes all the components for
video compression. Specifically, learning based optical flow estimation is
utilized to obtain the motion information and reconstruct the current frames.
Then we employ two auto-encoder style neural networks to compress the
corresponding motion and residual information. All the modules are jointly
learned through a single loss function, in which they collaborate with each
other by considering the trade-off between reducing the number of compression
bits and improving quality of the decoded video. Experimental results show that
the proposed approach can outperform the widely used video coding standard
H.264 in terms of PSNR and be even on par with the latest standard H.265 in
terms of MS-SSIM. Code is released at this https URL

本文提出了一种基于深度学习模型的视频压缩方法，其中利用神经网络的非线性表征能力和学习基于光流估计获取运动信息并重构当前帧的方法，同时采用两个 auto-encoder 风格的神经网络来压缩相应的运动和残差信息以共同优化所有组件，并通过单一损失函数来一起考虑减少压缩比特数和提高解码视频质量之间的权衡，实验证明该方法在 PSNR 方面优于广泛使用的 H.264 视频编码标准，在 MS-SSIM 方面甚至与最新的标准 H.265 相当。