Event cameras are novel sensors that report brightness changes in the form of
a stream of asynchronous "events" instead of intensity frames. They offer
significant advantages with respect to conventional cameras: high temporal
resolution, high dynamic range, and no motion blur. While the stream of events
encodes in principle the complete visual signal, the reconstruction of an
intensity image from a stream of events is an ill-posed problem in practice.
Existing reconstruction approaches are based on hand-crafted priors and strong
assumptions about the imaging process as well as the statistics of natural
images. In this work we propose to learn to reconstruct intensity images from
event streams directly from data instead of relying on any hand-crafted priors.
We propose a novel recurrent network to reconstruct videos from a stream of
events, and train it on a large amount of simulated event data. During training
we propose to use a perceptual loss to encourage reconstructions to follow
natural image statistics. We further extend our approach to synthesize color
images from color event streams. Our network surpasses state-of-the-art
reconstruction methods by a large margin in terms of image quality (> 20%),
while comfortably running in real-time. We show that the network is able to
synthesize high framerate videos (> 5,000 frames per second) of high-speed
phenomena (e.g. a bullet hitting an object) and is able to provide high dynamic
range reconstructions in challenging lighting conditions. We also demonstrate
the effectiveness of our reconstructions as an intermediate representation for
event data. We show that off-the-shelf computer vision algorithms can be
applied to our reconstructions for tasks such as object classification and
visual-inertial odometry and that this strategy consistently outperforms
algorithms that were specifically designed for event data.

本文提出了一种使用循环网络从事件流中直接学习重建图像的方法，并在此基础上扩展到从事件流合成彩色图像的方法，实验证明该网络在处理高速现象和挑战性照明条件下具有较好的图像质量和高动态范围重建，并作为事件数据的中间表示进行了分类和视觉惯性积分导航等应用。