We propose RTG-SLAM, a real-time 3D reconstruction system with an RGBD camera
for large-scale environments using Gaussian splatting. RTG-SLAM features a
compact Gaussian representation and a highly efficient on-the-fly Gaussian
optimization scheme. We force each Gaussian to be either opaque or nearly
transparent, with the opaque ones fitting the surface and dominant colors, and
transparent ones fitting residual colors. By rendering depth in a different way
from color rendering, we let a single opaque Gaussian well fit a local surface
region without the need of multiple overlapping Gaussians, hence largely
reducing the memory and computation cost. For on-the-fly Gaussian optimization,
we explicitly add Gaussians for three types of pixels per frame: newly
observed, with large color errors and with large depth errors. We also
categorize all Gaussians into stable and unstable ones, where the stable
Gaussians are expected to well fit previously observed RGBD images and
otherwise unstable. We only optimize the unstable Gaussians and only render the
pixels occupied by unstable Gaussians. In this way, both the number of
Gaussians to be optimized and pixels to be rendered are largely reduced, and
the optimization can be done in real time. We show real-time reconstructions of
a variety of real large scenes. Compared with the state-of-the-art NeRF-based
RGBD SLAM, our system achieves comparable high-quality reconstruction but with
around twice the speed and half the memory cost, and shows superior performance
in the realism of novel view synthesis and camera tracking accuracy.

通过使用高斯喷洒技术，我们提出了 RTG-SLAM，一个用于大规模环境的实时 3D 重建系统。与现有的基于 NeRF 的 RGBD SLAM 相比，我们的系统在速度和内存成本方面取得了可比的高质量重建，并且在新视角合成的逼真度和相机跟踪准确性上表现出更好的性能。