The reconstruction of indoor scenes from multi-view RGB images is challenging
due to the coexistence of flat and texture-less regions alongside delicate and
fine-grained regions. Recent methods leverage neural radiance fields aided by
predicted surface normal priors to recover the scene geometry. These methods
excel in producing complete and smooth results for floor and wall areas.
However, they struggle to capture complex surfaces with high-frequency
structures due to the inadequate neural representation and the inaccurately
predicted normal priors. To improve the capacity of the implicit
representation, we propose a hybrid architecture to represent low-frequency and
high-frequency regions separately. To enhance the normal priors, we introduce a
simple yet effective image sharpening and denoising technique, coupled with a
network that estimates the pixel-wise uncertainty of the predicted surface
normal vectors. Identifying such uncertainty can prevent our model from being
misled by unreliable surface normal supervisions that hinder the accurate
reconstruction of intricate geometries. Experiments on the benchmark datasets
show that our method significantly outperforms existing methods in terms of
reconstruction quality.

通过将低频和高频区域分别表示，并结合图像锐化和去噪技术以及估计像素级表面法线向量的网络，我们的混合架构和改进的正常先验可以显著提高室内场景的重建质量。

室内场景重建与细节增强的混合表达和法线先验

Indoor Scene Reconstruction with Fine-Grained Details Using Hybrid  Representation and Normal Prior Enhancement

Given a visual scene, humans have strong intuitions about how a scene can
evolve over time under given actions. The intuition, often termed visual
intuitive physics, is a critical ability that allows us to make effective plans
to manipulate the scene to achieve desired outcomes without relying on
extensive trial and error. In this paper, we present a framework capable of
learning 3D-grounded visual intuitive physics models from videos of complex
scenes with fluids. Our method is composed of a conditional Neural Radiance
Field (NeRF)-style visual frontend and a 3D point-based dynamics prediction
backend, using which we can impose strong relational and structural inductive
bias to capture the structure of the underlying environment. Unlike existing
intuitive point-based dynamics works that rely on the supervision of dense
point trajectory from simulators, we relax the requirements and only assume
access to multi-view RGB images and (imperfect) instance masks acquired using
color prior. This enables the proposed model to handle scenarios where accurate
point estimation and tracking are hard or impossible. We generate datasets
including three challenging scenarios involving fluid, granular materials, and
rigid objects in the simulation. The datasets do not include any dense particle
information so most previous 3D-based intuitive physics pipelines can barely
deal with that. We show our model can make long-horizon future predictions by
learning from raw images and significantly outperforms models that do not
employ an explicit 3D representation space. We also show that once trained, our
model can achieve strong generalization in complex scenarios under extrapolate
settings.

本论文提出了一种通过学习多视角 RGB 图像预测场景未来发展的 3D 模型的框架，并使用条件神经辐射场视觉前端和 3D 基点动态预测后端来捕获底层环境的结构以及强关系和结构归纳偏差，相较于先前的方法，我们的模型对准确的点估计和跟踪不再具有依赖性，可以应对较复杂的场景，并在长时间预测方面表现较好。