3D building reconstruction from imaging data is an important task for many
applications ranging from urban planning to reconnaissance. Modern Novel View
synthesis (NVS) methods like NeRF and Gaussian Splatting offer powerful
techniques for developing 3D models from natural 2D imagery in an unsupervised
fashion. These algorithms generally require input training views surrounding
the scene of interest, which, in the case of large buildings, is typically not
available across all camera elevations. In particular, the most readily
available camera viewpoints at scale across most buildings are at near-ground
(e.g., with mobile phones) and aerial (drones) elevations. However, due to the
significant difference in viewpoint between drone and ground image sets, camera
registration - a necessary step for NVS algorithms - fails. In this work we
propose a method, DRAGON, that can take drone and ground building imagery as
input and produce a 3D NVS model. The key insight of DRAGON is that
intermediate elevation imagery may be extrapolated by an NVS algorithm itself
in an iterative procedure with perceptual regularization, thereby bridging the
visual feature gap between the two elevations and enabling registration. We
compiled a semi-synthetic dataset of 9 large building scenes using Google Earth
Studio, and quantitatively and qualitatively demonstrate that DRAGON can
generate compelling renderings on this dataset compared to baseline strategies.

提出了 DRAGON 方法，能够将无人机和地面建筑图像作为输入，并生成 3D Novel View synthesis 模型，通过感知正则化的迭代过程填补了两种高度之间的视觉特征差距，从而实现了相机配准。在使用 Google Earth Studio 编译的 9 个大型建筑场景的半合成数据集上，定量和定性地证明了与基准策略相比，DRAGON 能够生成令人信服的渲染效果。

DRAGON: 无人机与地面高斯喷洒法用于 3D 建筑重建

DRAGON: Drone and Ground Gaussian Splatting for 3D Building  Reconstruction

Novel view synthesis from a sparse set of input images is a challenging
problem of great practical interest, especially when camera poses are absent or
inaccurate. Direct optimization of camera poses and usage of estimated depths
in neural radiance field algorithms usually do not produce good results because
of the coupling between poses and depths, and inaccuracies in monocular depth
estimation. In this paper, we leverage the recent 3D Gaussian splatting method
to develop a novel construct-and-optimize method for sparse view synthesis
without camera poses. Specifically, we construct a solution progressively by
using monocular depth and projecting pixels back into the 3D world. During
construction, we optimize the solution by detecting 2D correspondences between
training views and the corresponding rendered images. We develop a unified
differentiable pipeline for camera registration and adjustment of both camera
poses and depths, followed by back-projection. We also introduce a novel notion
of an expected surface in Gaussian splatting, which is critical to our
optimization. These steps enable a coarse solution, which can then be low-pass
filtered and refined using standard optimization methods. We demonstrate
results on the Tanks and Temples and Static Hikes datasets with as few as three
widely-spaced views, showing significantly better quality than competing
methods, including those with approximate camera pose information. Moreover,
our results improve with more views and outperform previous InstantNGP and
Gaussian Splatting algorithms even when using half the dataset.

通过利用 3D 高斯喷洒方法，我们开发了一种无需相机姿态的稀疏视图合成的新构建和优化方法，通过在构建过程中使用单目深度和将像素投影回 3D 世界来逐步构建解决方案，并通过检测训练视图与对应的渲染图像之间的 2D 对应关系，我们开发了一种统一可微的管道来完成相机注册和相机姿态与深度的调整，最后进行反投影。我们引入了高斯喷洒中预期表面的新概念，这对我们的优化至关重要。这些步骤使得我们能够获得一个粗糙的解决方案，然后可以使用标准优化方法进行低通滤波和细化。在 Tanks 和 Temples 以及 Static Hikes 数据集上，我们展示了仅使用三个广泛间隔的视图的结果，显示出明显优于竞争方法的质量，包括那些具有近似相机姿态信息的方法。此外，我们的结果在使用一半数据集时，随着视图数量的增加而改善，并且在使用 InstantNGP 和高斯喷洒算法时超过了先前的方法。

无需相机姿态的稀疏视角合成的构造优化方法

A Construct-Optimize Approach to Sparse View Synthesis without Camera  Pose

In this paper, we tackle the accurate and consistent Structure from Motion
(SfM) problem, in particular camera registration, far exceeding the memory of a
single computer in parallel. Different from the previous methods which
drastically simplify the parameters of SfM and sacrifice the accuracy of the
final reconstruction, we try to preserve the connectivities among cameras by
proposing a camera clustering algorithm to divide a large SfM problem into
smaller sub-problems in terms of camera clusters with overlapping. We then
exploit a hybrid formulation that applies the relative poses from local
incremental SfM into a global motion averaging framework and produce accurate
and consistent global camera poses. Our scalable formulation in terms of camera
clusters is highly applicable to the whole SfM pipeline including track
generation, local SfM, 3D point triangulation and bundle adjustment. We are
even able to reconstruct the camera poses of a city-scale data-set containing
more than one million high-resolution images with superior accuracy and
robustness evaluated on benchmark, Internet, and sequential data-sets.

本文提出一种可扩展的结构运动问题（SfM）求解方法，通过引入相机聚类算法将大型 SfM 问题划分为具有重叠的相机簇的子问题，并将局部增量 SfM 的相对位置应用到全局运动平均框架中以产生准确且一致的全局相机位姿。我们能够重建含有超过一百万张高分辨率图像的城市级数据集的相机位姿，在基准、Internet 和序列数据集上表现出卓越的准确性和鲁棒性。