Neuromorphic processors are well-suited for efficiently handling sparse
events from event-based cameras. However, they face significant challenges in
the growth of computing demand and hardware costs as the input resolution
increases. This paper proposes the Trainable Region-of-Interest Prediction
(TRIP), the first hardware-efficient hard attention framework for event-based
vision processing on a neuromorphic processor. Our TRIP framework actively
produces low-resolution Region-of-Interest (ROIs) for efficient and accurate
classification. The framework exploits sparse events' inherent low information
density to reduce the overhead of ROI prediction. We introduced extensive
hardware-aware optimizations for TRIP and implemented the hardware-optimized
algorithm on the SENECA neuromorphic processor. We utilized multiple
event-based classification datasets for evaluation. Our approach achieves
state-of-the-art accuracies in all datasets and produces reasonable ROIs with
varying locations and sizes. On the DvsGesture dataset, our solution requires
46x less computation than the state-of-the-art while achieving higher accuracy.
Furthermore, TRIP enables more than 2x latency and energy improvements on the
SENECA neuromorphic processor compared to the conventional solution.

本文提出了第一种硬件高效的基于神经形态处理器的硬注意力框架 Trainable Region-of-Interest Prediction (TRIP)，该框架通过利用稀疏事件的低信息密度来减少 ROI 预测的开销。在多个基于事件的分类数据集上，该方法达到了最先进的准确性，并在 DvsGesture 数据集上实现了 46 倍的计算量减少和更高的准确性，同时相比传统解决方案，在 SENECA 神经形态处理器上实现了超过 2 倍的延迟和能耗改进。

TRIP: 可训练的感兴趣区域预测技术，用于基于事件视觉的硬件高效神经形态处理

TRIP: Trainable Region-of-Interest Prediction for Hardware-Efficient  Neuromorphic Processing on Event-based Vision

Event cameras offer many advantages over standard cameras due to their
distinctive principle of operation: low power, low latency, high temporal
resolution and high dynamic range. Nonetheless, the success of many downstream
visual applications also hinges on an efficient and effective scene
representation, where Neural Radiance Field (NeRF) is seen as the leading
candidate. Such promise and potential of event cameras and NeRF inspired recent
works to investigate on the reconstruction of NeRF from moving event cameras.
However, these works are mainly limited in terms of the dependence on dense and
low-noise event streams, as well as generalization to arbitrary contrast
threshold values and camera speed profiles. In this work, we propose Robust
e-NeRF, a novel method to directly and robustly reconstruct NeRFs from moving
event cameras under various real-world conditions, especially from sparse and
noisy events generated under non-uniform motion. It consists of two key
components: a realistic event generation model that accounts for various
intrinsic parameters (e.g. time-independent, asymmetric threshold and
refractory period) and non-idealities (e.g. pixel-to-pixel threshold
variation), as well as a complementary pair of normalized reconstruction losses
that can effectively generalize to arbitrary speed profiles and intrinsic
parameter values without such prior knowledge. Experiments on real and novel
realistically simulated sequences verify our effectiveness. Our code, synthetic
dataset and improved event simulator are public.

提出了一种名为 Robust e-NeRF 的新方法，它可以在各种真实世界条件下直接且稳健地从移动事件相机中重建 NeRF，特别是从非均匀运动下生成的稀疏和嘈杂事件中。

稳健的 e-NeRF：在非均匀运动下从稀疏且嘈杂的事件中恢复 NeRF

Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion

When a camera travels across a 3D world, only a fraction of pixel value
changes; an event-based camera observes the change as sparse events. How can we
utilize sparse events for efficient recovery of the camera pose? We show that
we can recover the camera pose by minimizing the error between sparse events
and the temporal gradient of the scene represented as a neural radiance field
(NeRF). To enable the computation of the temporal gradient of the scene, we
augment NeRF's camera pose as a time function. When the input pose to the NeRF
coincides with the actual pose, the output of the temporal gradient of NeRF
equals the observed intensity changes on the event's points. Using this
principle, we propose an event-based camera pose tracking framework called
TeGRA which realizes the pose update by using the sparse event's observation.
To the best of our knowledge, this is the first camera pose estimation
algorithm using the scene's implicit representation and the sparse intensity
change from events.

使用神经辐射场 (NeRF) 作为场景的隐式表示，利用稀疏事件来更新摄像头位姿，实现了首个基于场景隐式表示和稀疏强度变化的摄像头姿态估计算法。