3D scene modeling techniques serve as the bedrocks in the geospatial
engineering and computer science, which drives many applications ranging from
automated driving, terrain mapping, navigation, virtual, augmented, mixed, and
extended reality (for gaming and movie industry etc.). This dissertation
presents a fraction of contributions that advances 3D scene modeling to its
state of the art, in the aspects of both appearance and geometry modeling. In
contrast to the prevailing deep learning methods, as a core contribution, this
thesis aims to develop algorithms that follow first principles, where
sophisticated physic-based models are introduced alongside with simpler
learning and inference tasks. The outcomes of these algorithms yield processes
that can consume much larger volume of data for highly accurate reconstructing
3D scenes at a scale without losing methodological generality, which are not
possible by contemporary complex-model based deep learning methods.
Specifically, the dissertation introduces three novel methodologies that
address the challenges of inferring appearance and geometry through
physics-based modeling.
Overall, the research encapsulated in this dissertation marks a series of
methodological triumphs in the processing of complex datasets. By navigating
the confluence of deep learning, computational geometry, and photogrammetry,
this work lays down a robust framework for future exploration and practical
application in the rapidly evolving field of 3D scene reconstruction. The
outcomes of these studies are evidenced through rigorous experiments and
comparisons with existing state-of-the-art methods, demonstrating the efficacy
and scalability of the proposed approaches.

本论文通过引入基于物理的模型，介绍了三种新方法来解决通过物理建模推断外观和几何的挑战，从而在处理复杂数据集方面取得了一系列方法上的胜利，为未来在 3D 场景重建的快速发展领域中的探索和实际应用奠定了一个强大的框架。

基于透视成像的可伸缩场景建模：基于物理的外观和几何推理

Scalable Scene Modeling from Perspective Imaging: Physics-based  Appearance and Geometry Inference

Recently, there has been growing interest in developing learning-based
methods to detect and utilize salient semi-global or global structures, such as
junctions, lines, planes, cuboids, smooth surfaces, and all types of
symmetries, for 3D scene modeling and understanding. However, the ground truth
annotations are often obtained via human labor, which is particularly
challenging and inefficient for such tasks due to the large number of 3D
structure instances (e.g., line segments) and other factors such as viewpoints
and occlusions. In this paper, we present a new synthetic dataset,
Structured3D, with the aim of providing large-scale photo-realistic images with
rich 3D structure annotations for a wide spectrum of structured 3D modeling
tasks. We take advantage of the availability of professional interior designs
and automatically extract 3D structures from them. We generate high-quality
images with an industry-leading rendering engine. We use our synthetic dataset
in combination with real images to train deep networks for room layout
estimation and demonstrate improved performance on benchmark datasets.

本文介绍了一种新的人工合成数据集 Structured3D，为各种结构化三维建模任务提供大规模照片级真实感图像和丰富的三维结构注释，此外并利用该数据集与真实图像来训练深度神经网络，显著提高了房间布局估计性能。

Structured3D: 用于结构化三维建模的大型真实照片数据集

Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling

Neural Processes combine the strengths of neural networks and Gaussian
processes to achieve both flexible learning and fast prediction in stochastic
processes. However, a large class of problems comprises underlying temporal
dependency structures in a sequence of stochastic processes that Neural
Processes (NP) do not explicitly consider. In this paper, we propose Sequential
Neural Processes (SNP) which incorporates a temporal state-transition model of
stochastic processes and thus extends its modeling capabilities to dynamic
stochastic processes. In applying SNP to dynamic 3D scene modeling, we
introduce the Temporal Generative Query Networks. To our knowledge, this is the
first 4D model that can deal with the temporal dynamics of 3D scenes. In
experiments, we evaluate the proposed methods in dynamic (non-stationary)
regression and 4D scene inference and rendering.

本文提出了一种 Sequential Neural Processes (SNP) 的方法，它通过引入一个时间状态转移模型来扩展 Neural Processes (NP) 的建模能力，以处理动态随机过程。作者还使用 Temporal Generative Query Networks 将 SNP 应用于动态三维场景建模，实现了可以处理 3D 场景时间动态的 4D 模型，并在实验中验证了该方法在动态回归和 4D 场景推断与渲染中的性能。