Event cameras are innovative neuromorphic sensors that asynchronously capture the scene dynamics. Due to the event-triggering mechanism, such cameras record event streams with much shorter response latency and higher intensity sensitivity compared to conventional cameras. On the basis of these features, previous works have attempted to reconstruct high dynamic range (HDR) videos from events, but have either suffered from unrealistic artifacts or failed to provide sufficiently high frame rates. In this paper, we present a recurrent convolutional neural network that reconstruct high-speed HDR videos from event sequences, with a key frame guidance to prevent potential error accumulation caused by the sparse event data. Additionally, to address the problem of severely limited real dataset, we develop a new optical system to collect a real-world dataset with paired high-speed HDR videos and event streams, facilitating future research in this field. Our dataset provides the first real paired dataset for event-to-HDR reconstruction, avoiding potential inaccuracies from simulation strategies. Experimental results demonstrate that our method can generate high-quality, high-speed HDR videos. We further explore the potential of our work in cross-camera reconstruction and downstream computer vision tasks, including object detection, panoramic segmentation, optical flow estimation, and monocular depth estimation under HDR scenarios.

本研究解决了现有事件相机在重建高动态范围（HDR）视频时面临的非现实伪影和帧率不足的问题。提出了一种递归卷积神经网络，利用关键帧引导来减小稀疏事件数据导致的误差累积，并开发了新的光学系统以收集配对的高速HDR视频和事件流数据集。实验表明，该方法能够生成高质量的高速HDR视频，并在跨相机重建和计算机视觉任务中展现潜力。

EventHDR：从事件到高速HDR视频及其扩展