Natural videos captured by consumer cameras often suffer from low framerate
and motion blur due to the combination of dynamic scene complexity, lens and
sensor imperfection, and less than ideal exposure setting. As a result,
computational methods that jointly perform video frame interpolation and
deblurring begin to emerge with the unrealistic assumption that the exposure
time is known and fixed. In this work, we aim ambitiously for a more realistic
and challenging task - joint video multi-frame interpolation and deblurring
under unknown exposure time. Toward this goal, we first adopt a variant of
supervised contrastive learning to construct an exposure-aware representation
from input blurred frames. We then train two U-Nets for intra-motion and
inter-motion analysis, respectively, adapting to the learned exposure
representation via gain tuning. We finally build our video reconstruction
network upon the exposure and motion representation by progressive
exposure-adaptive convolution and motion refinement. Extensive experiments on
both simulated and real-world datasets show that our optimized method achieves
notable performance gains over the state-of-the-art on the joint video x8
interpolation and deblurring task. Moreover, on the seemingly implausible x16
interpolation task, our method outperforms existing methods by more than 1.5 dB
in terms of PSNR.

本研究旨在有效地处理在未知曝光时间下所拍摄的视频，采取有监督对比学习构建曝光感知表征，使用两个 U-Net 进行运动分析，通过渐进式曝光自适应卷积和运动细化建立视频重建网络。在模拟和实际数据集上的实验表明，该优化方法在联合视频 x8 内插和去模糊任务上具有显着的性能提升，并且在看似不可能的 x16 内插任务上，该方法在 PSNR 方面比现有方法提高了 1.5 dB 以上。

未知曝光时间下的联合视频多帧插值和去模糊

Joint Video Multi-Frame Interpolation and Deblurring under Unknown  Exposure Time

Motion deblurring is a highly ill-posed problem due to the loss of motion
information in the blur degradation process. Since event cameras can capture
apparent motion with a high temporal resolution, several attempts have explored
the potential of events for guiding deblurring. These methods generally assume
that the exposure time is the same as the reciprocal of the video frame rate.
However, this is not true in real situations, and the exposure time might be
unknown and dynamically varies depending on the video shooting
environment(e.g., illumination condition). In this paper, we address the
event-guided motion deblurring assuming dynamically variable unknown exposure
time of the frame-based camera. To this end, we first derive a new formulation
for event-guided motion deblurring by considering the exposure and readout time
in the video frame acquisition process. We then propose a novel end-to-end
learning framework for event-guided motion deblurring. In particular, we design
a novel Exposure Time-based Event Selection(ETES) module to selectively use
event features by estimating the cross-modal correlation between the features
from blurred frames and the events. Moreover, we propose a feature fusion
module to fuse the selected features from events and blur frames effectively.
We conduct extensive experiments on various datasets and demonstrate that our
method achieves state-of-the-art performance.

本研究提出了一种使用事件相机进行运动去模糊的新的方法，并通过引入新的 Exposure Time-based Event Selection（ETES）模块和特征融合模块，提出了端到端的学习框架。在各个数据集上的实验表明，该方法可以达到最先进的性能水平。