Video frame interpolation is an important low-level vision task, which can
increase frame rate for more fluent visual experience. Existing methods have
achieved great success by employing advanced motion models and synthesis
networks. However, the spatial redundancy when synthesizing the target frame
has not been fully explored, that can result in lots of inefficient
computation. On the other hand, the computation compression degree in frame
interpolation is highly dependent on both texture distribution and scene
motion, which demands to understand the spatial-temporal information of each
input frame pair for a better compression degree selection. In this work, we
propose a novel two-stage frame interpolation framework termed WaveletVFI to
address above problems. It first estimates intermediate optical flow with a
lightweight motion perception network, and then a wavelet synthesis network
uses flow aligned context features to predict multi-scale wavelet coefficients
with sparse convolution for efficient target frame reconstruction, where the
sparse valid masks that control computation in each scale are determined by a
crucial threshold ratio. Instead of setting a fixed value like previous
methods, we find that embedding a classifier in the motion perception network
to learn a dynamic threshold for each sample can achieve more computation
reduction with almost no loss of accuracy. On the common high resolution and
animation frame interpolation benchmarks, proposed WaveletVFI can reduce
computation up to 40% while maintaining similar accuracy, making it perform
more efficiently against other state-of-the-arts. Code is available at
this https URL

提出了一种名为 WaveletVFI 的新型两阶段帧插值框架，通过轻量级运动感知网络估计中间光流，然后使用流对齐的上下文特征和稀疏卷积预测多尺度小波系数，以实现高效的目标帧重构，通过嵌入分类器学习动态阈值来控制计算，从而实现高达 40% 的计算减少，保持了相似的准确性。