Novel view synthesis from a sparse set of input images is a challenging
problem of great practical interest, especially when camera poses are absent or
inaccurate. Direct optimization of camera poses and usage of estimated depths
in neural radiance field algorithms usually do not produce good results because
of the coupling between poses and depths, and inaccuracies in monocular depth
estimation. In this paper, we leverage the recent 3D Gaussian splatting method
to develop a novel construct-and-optimize method for sparse view synthesis
without camera poses. Specifically, we construct a solution progressively by
using monocular depth and projecting pixels back into the 3D world. During
construction, we optimize the solution by detecting 2D correspondences between
training views and the corresponding rendered images. We develop a unified
differentiable pipeline for camera registration and adjustment of both camera
poses and depths, followed by back-projection. We also introduce a novel notion
of an expected surface in Gaussian splatting, which is critical to our
optimization. These steps enable a coarse solution, which can then be low-pass
filtered and refined using standard optimization methods. We demonstrate
results on the Tanks and Temples and Static Hikes datasets with as few as three
widely-spaced views, showing significantly better quality than competing
methods, including those with approximate camera pose information. Moreover,
our results improve with more views and outperform previous InstantNGP and
Gaussian Splatting algorithms even when using half the dataset.

通过利用 3D 高斯喷洒方法，我们开发了一种无需相机姿态的稀疏视图合成的新构建和优化方法，通过在构建过程中使用单目深度和将像素投影回 3D 世界来逐步构建解决方案，并通过检测训练视图与对应的渲染图像之间的 2D 对应关系，我们开发了一种统一可微的管道来完成相机注册和相机姿态与深度的调整，最后进行反投影。我们引入了高斯喷洒中预期表面的新概念，这对我们的优化至关重要。这些步骤使得我们能够获得一个粗糙的解决方案，然后可以使用标准优化方法进行低通滤波和细化。在 Tanks 和 Temples 以及 Static Hikes 数据集上，我们展示了仅使用三个广泛间隔的视图的结果，显示出明显优于竞争方法的质量，包括那些具有近似相机姿态信息的方法。此外，我们的结果在使用一半数据集时，随着视图数量的增加而改善，并且在使用 InstantNGP 和高斯喷洒算法时超过了先前的方法。