Event cameras sense intensity changes and have many advantages over conventional cameras. To take advantage of event cameras, some methods have been proposed to reconstruct intensity images from event streams. However, the outputs are still in low resolution (LR), noisy, and unrealistic. The low-quality outputs stem broader applications of event cameras, where high spatial resolution (HR) is needed as well as high temporal resolution, dynamic range, and no motion blur. We consider the problem of reconstructing and super-resolving intensity images from LR events, when no ground truth (GT) HR images and down-sampling kernels are available. To tackle the challenges, we propose a novel end-to-end pipeline that reconstructs LR images from event streams, enhances the image qualities and upsamples the enhanced images, called EventSR. For the absence of real GT images, our method is primarily unsupervised, deploying adversarial learning. To train EventSR, we create an open dataset including both real-world and simulated scenes. The use of both datasets boosts up the network performance, and the network architectures and various loss functions in each phase help improve the image qualities. The whole pipeline is trained in three phases. While each phase is mainly for one of the three tasks, the networks in earlier phases are fine-tuned by respective loss functions in an end-to-end manner. Experimental results show that EventSR reconstructs high-quality SR images from events for both simulated and real-world data.

本文提出了一种基于事件流的、重建低分辨率图像、提高图像质量并提高分辨率的新方法——EventSR。该方法是基于非监督式对抗学习的，对缺乏真实高分辨率图像的情况下进行重建，并利用了包括真实和模拟场景在内的数据集来提高网络性能。实验结果表明，EventSR可以在仿真和实际数据场景下从事件中重建出高质量的高分辨率图像。

EventSR: 通过端到端的对抗学习, 从异步事件到图像重建、修复和超分辨率