The field of 3D reconstruction from images has rapidly evolved in the past
few years, first with the introduction of Neural Radiance Field (NeRF) and more
recently with 3D Gaussian Splatting (3DGS). The latter provides a significant
edge over NeRF in terms of the training and inference speed, as well as the
reconstruction quality. Although 3DGS works well for dense input images, the
unstructured point-cloud like representation quickly overfits to the more
challenging setup of extremely sparse input images (e.g., 3 images), creating a
representation that appears as a jumble of needles from novel views. To address
this issue, we propose regularized optimization and depth-based initialization.
Our key idea is to introduce a structured Gaussian representation that can be
controlled in 2D image space. We then constraint the Gaussians, in particular
their position, and prevent them from moving independently during optimization.
Specifically, we introduce single and multiview constraints through an implicit
convolutional decoder and a total variation loss, respectively. With the
coherency introduced to the Gaussians, we further constrain the optimization
through a flow-based loss function. To support our regularized optimization, we
propose an approach to initialize the Gaussians using monocular depth estimates
at each input view. We demonstrate significant improvements compared to the
state-of-the-art sparse-view NeRF-based approaches on a variety of scenes.

通过结构化的高斯表示和正则化优化，以及深度基于初始化来解决三维重建中稀疏输入图像的挑战，提高了与现有方法相比的重建质量。