This paper proposes a low latency neural network architecture for event-based
dense prediction tasks. Conventional architectures encode entire scene contents
at a fixed rate regardless of their temporal characteristics. Instead, the
proposed network encodes contents at a proper temporal scale depending on its
movement speed. We achieve this by constructing temporal hierarchy using
stacked latent memories that operate at different rates. Given low latency
event steams, the multi-level memories gradually extract dynamic to static
scene contents by propagating information from the fast to the slow memory
modules. The architecture not only reduces the redundancy of conventional
architectures but also exploits long-term dependencies. Furthermore, an
attention-based event representation efficiently encodes sparse event streams
into the memory cells. We conduct extensive evaluations on three event-based
dense prediction tasks, where the proposed approach outperforms the existing
methods on accuracy and latency, while demonstrating effective event and image
fusion capabilities. The code is available at this https URL

该论文提出了一种用于事件类密集预测任务的低延迟神经网络架构，该架构通过构建分层的时间记忆来在适当的时间尺度上编码内容，并通过基于注意力机制的事件表示将稀疏事件流编码为记忆单元，有效地提高了准确性和性能。