In this work, we use multi-view aerial images to reconstruct the geometry,
lighting, and material of facades using neural signed distance fields (SDFs).
Without the requirement of complex equipment, our method only takes simple RGB
images captured by a drone as inputs to enable physically based and
photorealistic novel-view rendering, relighting, and editing. However, a
real-world facade usually has complex appearances ranging from diffuse rocks
with subtle details to large-area glass windows with specular reflections,
making it hard to attend to everything. As a result, previous methods can
preserve the geometry details but fail to reconstruct smooth glass windows or
verse vise. In order to address this challenge, we introduce three spatial- and
semantic-adaptive optimization strategies, including a semantic regularization
approach based on zero-shot segmentation techniques to improve material
consistency, a frequency-aware geometry regularization to balance surface
smoothness and details in different surfaces, and a visibility probe-based
scheme to enable efficient modeling of the local lighting in large-scale
outdoor environments. In addition, we capture a real-world facade aerial 3D
scanning image set and corresponding point clouds for training and
benchmarking. The experiment demonstrates the superior quality of our method on
facade holistic inverse rendering, novel view synthesis, and scene editing
compared to state-of-the-art baselines.

使用多视角航拍图像，通过神经符号距离场重建立面的几何形状、光照和材质，从而实现基于物理和照片般逼真的新视角渲染、重照和编辑。通过引入三种自适应优化策略，包括基于零样本分割技术的语义正则化方法来改善材料一致性，基于频率感知的几何正则化来平衡不同表面的表面光滑性和细节，以及基于可见性探针的方案来实现大规模户外环境中局部照明的有效建模。实验结果通过与最先进的基线方法进行比较，展示了我们的方法在立面整体反渲染、新视角合成和场景编辑方面的卓越质量。