Learning approaches have shown great success in the task of super-resolving an image given a low resolution input. Video super-resolution aims for exploiting additionally the information from multiple images. Typically, the images are related via optical flow and consecutive image warping. In this paper, we provide an end-to-end video super-resolution network that, in contrast to previous works, includes the estimation of optical flow in the overall network architecture. We analyze the usage of optical flow for video super-resolution and find that common off-the-shelf image warping does not allow video super-resolution to benefit much from optical flow. We rather propose an operation for motion compensation that performs warping from low to high resolution directly. We show that with this network configuration, video super-resolution can benefit from optical flow and we obtain state-of-the-art results on the popular test sets. We also show that the processing of whole images rather than independent patches is responsible for a large increase in accuracy.

本文提出了一种端到端的视频超分辨率网络，它包括了在整个网络架构中估计光流，研究了光流在视频超分辨率中的使用，并发现常见的离线图像变形不利于视频超分辨率，因此提出了一种运动补偿操作。通过对整张图像进行处理而不是独立的图像块，可以大幅提高精度。结果表明，使用该网络架构可以受益于光流，获得了在流行测试集上的最佳结果。

运动补偿视频超分辨率的端到端学习