When a very fast dynamic event is recorded with a low-framerate camera, the
resulting video suffers from severe motion blur (due to exposure time) and
motion aliasing (due to low sampling rate in time). True Temporal
Super-Resolution (TSR) is more than just Temporal-Interpolation (increasing
framerate). It can also recover new high temporal frequencies beyond the
temporal Nyquist limit of the input video, thus resolving both motion-blur and
motion-aliasing effects that temporal frame interpolation (as sophisticated as
it maybe) cannot undo. In this paper we propose a "Deep Internal Learning"
approach for true TSR. We train a video-specific CNN on examples extracted
directly from the low-framerate input video. Our method exploits the strong
recurrence of small space-time patches inside a single video sequence, both
within and across different spatio-temporal scales of the video. We further
observe (for the first time) that small space-time patches recur also
across-dimensions of the video sequence - i.e., by swapping the spatial and
temporal dimensions. In particular, the higher spatial resolution of video
frames provides strong examples as to how to increase the temporal resolution
of that video. Such internal video-specific examples give rise to strong
self-supervision, requiring no data but the input video itself. This results in
Zero-Shot Temporal-SR of complex videos, which removes both motion blur and
motion aliasing, outperforming previous supervised methods trained on external
video datasets.

本文提出了一种基于 Deep Internal Learning 的 Temporal Super-Resolution 方法，利用小空间时间片段内部的强烈重复性，从而在不使用任何其他数据集的情况下，实现了复杂视频的零成本 Temporal Super-Resolution，有效消除运动模糊和运动混叠效果，并超越了以往依赖于外部数据集进行训练的监督方法。

跨尺度和跨维度：使用深度内部学习进行时间超分辨率

Across Scales & Across Dimensions: Temporal Super-Resolution using Deep  Internal Learning

Super resolution (SR) methods typically assume that the low-resolution (LR)
image was downscaled from the unknown high-resolution (HR) image by a fixed
'ideal' downscaling kernel (e.g. Bicubic downscaling). However, this is rarely
the case in real LR images, in contrast to synthetically generated SR datasets.
When the assumed downscaling kernel deviates from the true one, the performance
of SR methods significantly deteriorates. This gave rise to Blind-SR - namely,
SR when the downscaling kernel ("SR-kernel") is unknown. It was further shown
that the true SR-kernel is the one that maximizes the recurrence of patches
across scales of the LR image. In this paper we show how this powerful
cross-scale recurrence property can be realized using Deep Internal Learning.
We introduce "KernelGAN", an image-specific Internal-GAN, which trains solely
on the LR test image at test time, and learns its internal distribution of
patches. Its Generator is trained to produce a downscaled version of the LR
test image, such that its Discriminator cannot distinguish between the patch
distribution of the downscaled image, and the patch distribution of the
original LR image. The Generator, once trained, constitutes the downscaling
operation with the correct image-specific SR-kernel. KernelGAN is fully
unsupervised, requires no training data other than the input image itself, and
leads to state-of-the-art results in Blind-SR when plugged into existing SR
algorithms.

介绍了一种新的超分辨率方法 —— 基于 Deep Internal Learning 的 KernelGAN。该方法可以在不知道下采样核（SR-kernel）的情况下进行盲超分辨率（Blind-SR），不需要数据集，实现了最先进的效果。