This paper presents a robust approach for a visual parallel tracking and mapping (PTAM) system that excels in challenging environments. Our proposed method combines the strengths of heterogeneous multi-modal visual sensors, including stereo event-based and frame-based sensors, in a unified reference frame through a novel spatio-temporal synchronization of stereo visual frames and stereo event streams. We employ deep learning-based feature extraction and description for estimation to enhance robustness further. We also introduce an end-to-end parallel tracking and mapping optimization layer complemented by a simple loop-closure algorithm for efficient SLAM behavior. Through comprehensive experiments on both small-scale and large-scale real-world sequences of VECtor and TUM-VIE benchmarks, our proposed method (DH-PTAM) demonstrates superior performance compared to state-of-the-art methods in terms of robustness and accuracy in adverse conditions. Our implementation's research-based Python API is publicly available on GitHub for further research and development: https://github.com/AbanobSoliman/DH-PTAM.

本文介绍了一种适用于复杂环境下的视觉平行跟踪和建图系统的稳健方法，该方法通过新颖的时空同步实现了异构多模式视觉传感器，包括立体事件和帧传感器，以一个统一的参考坐标系。此外，我们采用了基于深度学习的特征提取和描述方法来增强稳健性，并引入了一个端到端的并行跟踪和映射优化层，结合一个简单的回环算法以实现高效的SLAM行为。通过对VECtor和TUM-VIE基准的小尺度和大尺度现实世界序列进行的全面实验，我们的方法（DH-PTAM）在恶劣条件下表现出优越的性能，并且在稳健性和准确性方面优于现有方法。我们实现的研究基于的Python API可以在GitHub上公开获取，供进一步研究和开发使用。

DH-PTAM: 一种深度立体事件帧并行跟踪与建图系统