Video super-resolution (SR) aims at generating a sequence of high-resolution (HR) frames with plausible and temporally consistent details from their low-resolution (LR) counterparts. The key challenge for video SR lies in the effective exploitation of temporal dependency between consecutive frames. Existing deep learning based methods commonly estimate optical flows between LR frames to provide temporal dependency. However, the resolution conflict between LR optical flows and HR outputs hinders the recovery of fine details. In this paper, we propose an end-to-end video SR network to super-resolve both optical flows and images. Optical flow SR from LR frames provides accurate temporal dependency and ultimately improves video SR performance. Specifically, we first propose an optical flow reconstruction network (OFRnet) to infer HR optical flows in a coarse-to-fine manner. Then, motion compensation is performed using HR optical flows to encode temporal dependency. Finally, compensated LR inputs are fed to a super-resolution network (SRnet) to generate SR results. Extensive experiments have been conducted to demonstrate the effectiveness of HR optical flows for SR performance improvement. Comparative results on the Vid4 and DAVIS-10 datasets show that our network achieves the state-of-the-art performance.

本文提出了一种用于视频超分辨率的端到端深度学习网络，其中通过光流重建和图像超分辨率的结合，采用粗到细的方式实现了高分辨率光流的重建和精细细节的恢复，以解决视频SR中针对低分辨率光流与高分辨率输出间存在的分辨率矛盾问题，并通过广泛实验证明了其在提高视频SR性能方面的有效性及其在Vid4和DAVIS-10数据集上的最新成果。

使用高分辨率光流估计的深度视频超分辨率