Event cameras are vision sensors that record asynchronous streams of per-pixel brightness changes, referred to as "events". They have appealing advantages over frame-based cameras for computer vision, including high temporal resolution, high dynamic range, and no motion blur. Due to the sparse, non-uniform spatiotemporal layout of the event signal, pattern recognition algorithms typically aggregate events into a grid-based representation and subsequently process it by a standard vision pipeline, e.g., Convolutional Neural Network (CNN). In this work, we introduce a general framework to convert event streams into grid-based representations through a sequence of differentiable operations. Our framework comes with two main ad-vantages: (i) allows learning the input event representation together with the task dedicated network in an end to end manner, and (ii) lays out a taxonomy that unifies the majority of extant event representations in the literature and identifies novel ones. Empirically, we show that our approach to learning the event representation end-to-end yields an improvement of approximately 12% on optical flow estimation and object recognition over state-of-the-art methods.

本文介绍了将事件流转化成基于网格的表示的通用框架，其具有两个主要优势：（i）允许端到端学习输入事件表示，（ii）提供了一个统一记录大多数事件表示的分类法，并识别出新的事件表示方法。通过实验证明，我们的端到端学习事件表示的方法相对于现有技术方法，在光流估计和目标识别上都有约12％的改进。

异步事件数据表示的端到端学习