Recently, video frame interpolation using a combination of frame- and
event-based cameras has surpassed traditional image-based methods both in terms
of performance and memory efficiency. However, current methods still suffer
from (i) brittle image-level fusion of complementary interpolation results,
that fails in the presence of artifacts in the fused image, (ii) potentially
temporally inconsistent and inefficient motion estimation procedures, that run
for every inserted frame and (iii) low contrast regions that do not trigger
events, and thus cause events-only motion estimation to generate artifacts.
Moreover, previous methods were only tested on datasets consisting of planar
and faraway scenes, which do not capture the full complexity of the real world.
In this work, we address the above problems by introducing multi-scale
feature-level fusion and computing one-shot non-linear inter-frame motion from
events and images, which can be efficiently sampled for image warping. We also
collect the first large-scale events and frames dataset consisting of more than
100 challenging scenes with depth variations, captured with a new experimental
setup based on a beamsplitter. We show that our method improves the
reconstruction quality by up to 0.2 dB in terms of PSNR and up to 15% in LPIPS
score.

本文提出了基于多尺度特征融合技术和一次非线性帧间运动计算的视频帧插值方法，利用新型实验装置捕捉深度变化的 100 多个具有挑战性的场景，对比实验表明我们的方法在 PSNR 和 LPIPS 得分上分别提高了 0.2dB 和 15%。