Generating non-existing frames from a consecutive video sequence has been an
interesting and challenging problem in the video processing field. Typical
kernel-based interpolation methods predict pixels with a single convolution
process that convolves source frames with spatially adaptive local kernels,
which circumvents the time-consuming, explicit motion estimation in the form of
optical flow. However, when scene motion is larger than the pre-defined kernel
size, these methods are prone to yield less plausible results. In addition,
they cannot directly generate a frame at an arbitrary temporal position because
the learned kernels are tied to the midpoint in time between the input frames.
In this paper, we try to solve these problems and propose a novel non-flow
kernel-based approach that we refer to as enhanced deformable separable
convolution (EDSC) to estimate not only adaptive kernels, but also offsets,
masks and biases to make the network obtain information from non-local
neighborhood. During the learning process, different intermediate time step can
be involved as a control variable by means of an extension of coord-conv trick,
allowing the estimated components to vary with different input temporal
information. This makes our method capable to produce multiple in-between
frames. Furthermore, we investigate the relationships between our method and
other typical kernel- and flow-based methods. Experimental results show that
our method performs favorably against the state-of-the-art methods across a
broad range of datasets. Code will be publicly available on URL:
https://github.com/Xianhang/EDSC-pytorch.

该论文提出了一种增强的可变形可分离卷积（EDSC）来生成视频序列的中间帧，并能够以任意时间位置生成任意帧。通过学习自适应卷积核、偏移量、掩码和偏置等多个组件，能够从非局部邻域获取信息，并且通过扩展的 coord-conv 技巧，能够控制中间时间步来生成多个中间帧。实验结果表明，该方法在广泛的数据集范围内优于现有方法。