We present a real-time visual-inertial dense mapping method capable of
performing incremental 3D mesh reconstruction with high quality using only
sequential monocular images and inertial measurement unit (IMU) readings. 6-DoF
camera poses are estimated by a robust feature-based visual-inertial odometry
(VIO), which also generates noisy sparse 3D map points as a by-product. We
propose a sparse point aided multi-view stereo neural network (SPA-MVSNet) that
can effectively leverage the informative but noisy sparse points from the VIO
system. The sparse depth from VIO is firstly completed by a single-view depth
completion network. This dense depth map, although naturally limited in
accuracy, is then used as a prior to guide our MVS network in the cost volume
generation and regularization for accurate dense depth prediction. Predicted
depth maps of keyframe images by the MVS network are incrementally fused into a
global map using TSDF-Fusion. We extensively evaluate both the proposed
SPA-MVSNet and the entire visual-inertial dense mapping system on several
public datasets as well as our own dataset, demonstrating the system's
impressive generalization capabilities and its ability to deliver high-quality
3D mesh reconstruction online. Our proposed dense mapping system achieves a
39.7% improvement in F-score over existing systems when evaluated on the
challenging scenarios of the EuRoC dataset. We plan to release the code of this
work upon acceptance.

我们提出了一种实时的基于视觉惯性的稠密建图方法，可使用顺序单眼图像和惯性测量单元（IMU）读数进行增量式三维网格重建，并提出了稀疏点辅助的多视角深度学习神经网络 (SPA-MVSNet)，即使在噪声密集的情况下也能实现高质量的 3D 网格重建。