Vision-based localization is a cost-effective and thus attractive solution
for many intelligent mobile platforms. However, its accuracy and especially
robustness still suffer from low illumination conditions, illumination changes,
and aggressive motion. Event-based cameras are bio-inspired visual sensors that
perform well in HDR conditions and have high temporal resolution, and thus
provide an interesting alternative in such challenging scenarios. While purely
event-based solutions currently do not yet produce satisfying mapping results,
the present work demonstrates the feasibility of purely event-based tracking if
an alternative sensor is permitted for mapping. The method relies on geometric
3D-2D registration of semi-dense maps and events, and achieves highly reliable
and accurate cross-modal tracking results. Practically relevant scenarios are
given by depth camera-supported tracking or map-based localization with a
semi-dense map prior created by a regular image-based visual SLAM or
structure-from-motion system. Conventional edge-based 3D-2D alignment is
extended by a novel polarity-aware registration that makes use of signed
time-surface maps (STSM) obtained from event streams. We furthermore introduce
a novel culling strategy for occluded points. Both modifications increase the
speed of the tracker and its robustness against occlusions or large view-point
variations. The approach is validated on many real datasets covering the
above-mentioned challenging conditions, and compared against similar solutions
realised with regular cameras.

使用基于事件的相机进行视觉定位，通过几何三维到二维的映射和极性感知的注册方法实现高可靠性和准确性的多模态跟踪，同时引入新的遮挡点筛选策略提高追踪器的速度和鲁棒性。

具挑战条件下的跨模态半密集 6 自由度事件相机跟踪

Cross-Modal Semi-Dense 6-DoF Tracking of an Event Camera in Challenging  Conditions

Depth completion recovers a dense depth map from sensor measurements. Current
methods are mostly tailored for very sparse depth measurements from LiDARs in
outdoor settings, while for indoor scenes Time-of-Flight (ToF) or structured
light sensors are mostly used. These sensors provide semi-dense maps, with
dense measurements in some regions and almost empty in others. We propose a new
model that takes into account the statistical difference between such regions.
Our main contribution is a new decoder modulation branch added to the
encoder-decoder architecture. The encoder extracts features from the
concatenated RGB image and raw depth. Given the mask of missing values as
input, the proposed modulation branch controls the decoding of a dense depth
map from these features differently for different regions. This is implemented
by modifying the spatial distribution of output signals inside the decoder via
Spatially-Adaptive Denormalization (SPADE) blocks. Our second contribution is a
novel training strategy that allows us to train on a semi-dense sensor data
when the ground truth depth map is not available. Our model achieves the state
of the art results on indoor Matterport3D dataset. Being designed for
semi-dense input depth, our model is still competitive with LiDAR-oriented
approaches on the KITTI dataset. Our training strategy significantly improves
prediction quality with no dense ground truth available, as validated on the
NYUv2 dataset.

该研究提出了一种基于编码器 - 解码器架构的新型深度完成模型，利用重新定位输出信号内的空间分布，实现根据不同地区的控制，通过 Spatially-Adaptive Denormalization（SPADE）块来控制特定区域的密集度 map 解码，有效提高模型预测质量，主要贡献包含了编码器 - 解码器架构之外的特定部分设计和新颖的训练策略。