Dynamic vision sensors or event cameras provide rich complementary
information for video frame interpolation. Existing state-of-the-art methods
follow the paradigm of combining both synthesis-based and warping networks.
However, few of those methods fully respect the intrinsic characteristics of
events streams. Given that event cameras only encode intensity changes and
polarity rather than color intensities, estimating optical flow from events is
arguably more difficult than from RGB information. We therefore propose to
incorporate RGB information in an event-guided optical flow refinement
strategy. Moreover, in light of the quasi-continuous nature of the time signals
provided by event cameras, we propose a divide-and-conquer strategy in which
event-based intermediate frame synthesis happens incrementally in multiple
simplified stages rather than in a single, long stage. Extensive experiments on
both synthetic and real-world datasets show that these modifications lead to
more reliable and realistic intermediate frame results than previous video
frame interpolation methods. Our findings underline that a careful
consideration of event characteristics such as high temporal density and
elevated noise benefits interpolation accuracy.

利用动态视觉传感器或事件相机进行视频帧插值，结合 RGB 信息的事件引导光流细化策略和分阶段的中间帧合成策略，可以获得比先前的视频帧插值方法更可靠和更真实的中间帧结果。

重新审视基于事件的视频帧插值

Revisiting Event-based Video Frame Interpolation

Video frame interpolation, which aims to synthesize non-exist intermediate
frames in a video sequence, is an important research topic in computer vision.
Existing video frame interpolation methods have achieved remarkable results
under specific assumptions, such as instant or known exposure time. However, in
complicated real-world situations, the temporal priors of videos, i.e. frames
per second (FPS) and frame exposure time, may vary from different camera
sensors. When test videos are taken under different exposure settings from
training ones, the interpolated frames will suffer significant misalignment
problems. In this work, we solve the video frame interpolation problem in a
general situation, where input frames can be acquired under uncertain exposure
(and interval) time. Unlike previous methods that can only be applied to a
specific temporal prior, we derive a general curvilinear motion trajectory
formula from four consecutive sharp frames or two consecutive blurry frames
without temporal priors. Moreover, utilizing constraints within adjacent motion
trajectories, we devise a novel optical flow refinement strategy for better
interpolation results. Finally, experiments demonstrate that one well-trained
model is enough for synthesizing high-quality slow-motion videos under
complicated real-world situations. Codes are available on
this https URL

本文是一篇关于计算机视觉中视频插帧的研究，提出了一种基于不确定帧率和曝光时间的一般情况下视频插帧问题的解决方案。使用四个连续的清晰帧或两个连续的模糊帧，通过得出一般曲线运动轨迹公式的方法解决了现有视频帧插值方法只适用于特定时间先验的问题，并通过相邻运动轨迹中的约束定义了一种新的光流细化策略以提高插值结果。实验表明，一个训练良好的模型足以在复杂的实际情况下合成高质量的慢动作录像。