Video frame interpolation (VFI) is the task that synthesizes the intermediate
frame given two consecutive frames. Most of the previous studies have focused
on appropriate frame warping operations and refinement modules for the warped
frames. These studies have been conducted on natural videos containing only
continuous motions. However, many practical videos contain various unnatural
objects with discontinuous motions such as logos, user interfaces and
subtitles. We propose three techniques to make the existing deep learning-based
VFI architectures robust to these elements. First is a novel data augmentation
strategy called figure-text mixing (FTM) which can make the models learn
discontinuous motions during training stage without any extra dataset. Second,
we propose a simple but effective module that predicts a map called
discontinuity map (D-map), which densely distinguishes between areas of
continuous and discontinuous motions. Lastly, we propose loss functions to give
supervisions of the discontinuous motion areas which can be applied along with
FTM and D-map. We additionally collect a special test benchmark called
Graphical Discontinuous Motion (GDM) dataset consisting of some mobile games
and chatting videos. Applied to the various state-of-the-art VFI networks, our
method significantly improves the interpolation qualities on the videos from
not only GDM dataset, but also the existing benchmarks containing only
continuous motions such as Vimeo90K, UCF101, and DAVIS.

本文提出了三种技术来加强现有深度学习视频插帧 (VFI) 架构对不连续运动的鲁棒性。第一种是一种称为 figure-text mixing (FTM) 的新颖数据增强策略，无需额外数据集便可在训练时使模型学习不连续运动。第二种是一种简单而有效的模块，用于预测称为 discontinuity map (D-map) 的地图，密集地区分持续和不连续运动区域。最后，我们提出了损失函数，以对不连续运动区域进行监督。应用于各种先进的 VFI 网络，我们的方法显著提高了插帧质量，不仅适用于 GDM 数据集，还包括只包含连续运动的现有基准数据集，例如 Vimeo90K、UCF101 和 DAVIS。