In this paper, we study a practical space-time video super-resolution (STVSR)
problem which aims at generating a high-framerate high-resolution sharp video
from a low-framerate low-resolution blurry video. Such problem often occurs
when recording a fast dynamic event with a low-framerate and low-resolution
camera, and the captured video would suffer from three typical issues: i)
motion blur occurs due to object/camera motions during exposure time; ii)
motion aliasing is unavoidable when the event temporal frequency exceeds the
Nyquist limit of temporal sampling; iii) high-frequency details are lost
because of the low spatial sampling rate. These issues can be alleviated by a
cascade of three separate sub-tasks, including video deblurring, frame
interpolation, and super-resolution, which, however, would fail to capture the
spatial and temporal correlations among video sequences. To address this, we
propose an interpretable STVSR framework by leveraging both model-based and
learning-based methods. Specifically, we formulate STVSR as a joint video
deblurring, frame interpolation, and super-resolution problem, and solve it as
two sub-problems in an alternate way. For the first sub-problem, we derive an
interpretable analytical solution and use it as a Fourier data transform layer.
Then, we propose a recurrent video enhancement layer for the second sub-problem
to further recover high-frequency details. Extensive experiments demonstrate
the superiority of our method in terms of quantitative metrics and visual
quality.

本文提出了一种可行的空时视频超分辨率（STVSR）框架，以解决鲁棒的视频采集中模糊、模糊和低分辨率等挑战，提供优异的定量测量和视觉质量表现。