In the animation industry, cartoon videos are usually produced at low frame
rate since hand drawing of such frames is costly and time-consuming. Therefore,
it is desirable to develop computational models that can automatically
interpolate the in-between animation frames. However, existing video
interpolation methods fail to produce satisfying results on animation data.
Compared to natural videos, animation videos possess two unique characteristics
that make frame interpolation difficult: 1) cartoons comprise lines and smooth
color pieces. The smooth areas lack textures and make it difficult to estimate
accurate motions on animation videos. 2) cartoons express stories via
exaggeration. Some of the motions are non-linear and extremely large. In this
work, we formally define and study the animation video interpolation problem
for the first time. To address the aforementioned challenges, we propose an
effective framework, AnimeInterp, with two dedicated modules in a
coarse-to-fine manner. Specifically, 1) Segment-Guided Matching resolves the
"lack of textures" challenge by exploiting global matching among color pieces
that are piece-wise coherent. 2) Recurrent Flow Refinement resolves the
"non-linear and extremely large motion" challenge by recurrent predictions
using a transformer-like architecture. To facilitate comprehensive training and
evaluations, we build a large-scale animation triplet dataset, ATD-12K, which
comprises 12,000 triplets with rich annotations. Extensive experiments
demonstrate that our approach outperforms existing state-of-the-art
interpolation methods for animation videos. Notably, AnimeInterp shows
favorable perceptual quality and robustness for animation scenarios in the
wild. The proposed dataset and code are available at
this https URL

本研究针对动画视频插帧问题，提出 AnimeInterp 框架，包括 Segment-Guided Matching 和 Recurrent Flow Refinement 模块，能够有效解决动画视频的颜色匹配和非线性动作问题，该方法在大规模动画三重奏数据集 ATD-12K 上表现优于现有最先进的插帧方法，并具有较优的视觉质量和鲁棒性。