How do we infer a 3D scene from a single image in the presence of corruptions
like rain, snow or fog? Straightforward domain randomization relies on knowing
the family of corruptions ahead of time. Here, we propose a Bayesian
approach-dubbed robust inverse graphics (RIG)-that relies on a strong scene
prior and an uninformative uniform corruption prior, making it applicable to a
wide range of corruptions. Given a single image, RIG performs posterior
inference jointly over the scene and the corruption. We demonstrate this idea
by training a neural radiance field (NeRF) scene prior and using a secondary
NeRF to represent the corruptions over which we place an uninformative prior.
RIG, trained only on clean data, outperforms depth estimators and alternative
NeRF approaches that perform point estimation instead of full inference. The
results hold for a number of scene prior architectures based on normalizing
flows and diffusion models. For the latter, we develop reconstruction-guidance
with auxiliary latents (ReGAL)-a diffusion conditioning algorithm that is
applicable in the presence of auxiliary latent variables such as the
corruption. RIG demonstrates how scene priors can be used beyond generation
tasks.

通过使用具有强大场景先验和无信息均匀损坏先验的贝叶斯方法（RIG），在给定单一图像的情况下，我们能够推断出含有各种污染物（如雨、雪或雾）的 3D 场景，并展示了 RIG 是如何超越点估计而优于深度估计器和其他性能的 NeRF 方法的。

稳健逆图形推理的概率推断

Robust Inverse Graphics via Probabilistic Inference

High-fidelity 3D scene reconstruction has been substantially advanced by
recent progress in neural fields. However, most existing methods train a
separate network from scratch for each individual scene. This is not scalable,
inefficient, and unable to yield good results given limited views. While
learning-based multi-view stereo methods alleviate this issue to some extent,
their multi-view setting makes it less flexible to scale up and to broad
applications. Instead, we introduce training generalizable Neural Fields
incorporating scene Priors (NFPs). The NFP network maps any single-view RGB-D
image into signed distance and radiance values. A complete scene can be
reconstructed by merging individual frames in the volumetric space WITHOUT a
fusion module, which provides better flexibility. The scene priors can be
trained on large-scale datasets, allowing for fast adaptation to the
reconstruction of a new scene with fewer views. NFP not only demonstrates SOTA
scene reconstruction performance and efficiency, but it also supports
single-image novel-view synthesis, which is underexplored in neural fields.
More qualitative results are available at:
this https URL

在高保真度 3D 场景重建方面，神经场的最新进展已经有了实质性的提升。然而，大多数现有方法为每个独立场景训练单独的神经网络，这不可扩展、低效且对有限视角下的结果不理想。本研究引入了训练可推广的神经场，将场景先验结合其中，从而更好地解决上述问题，并支持单张图像的新视角合成。

使用场景先验的通用神经场进行三维重建

3D Reconstruction with Generalizable Neural Fields using Scene Priors

Previous online 3D dense reconstruction methods struggle to achieve the
balance between memory storage and surface quality, largely due to the usage of
stagnant underlying geometry representation, such as TSDF (truncated signed
distance functions) or surfels, without any knowledge of the scene priors. In
this paper, we present DI-Fusion (Deep Implicit Fusion), based on a novel 3D
representation, i.e. Probabilistic Local Implicit Voxels (PLIVoxs), for online
3D reconstruction with a commodity RGB-D camera. Our PLIVox encodes scene
priors considering both the local geometry and uncertainty parameterized by a
deep neural network. With such deep priors, we are able to perform online
implicit 3D reconstruction achieving state-of-the-art camera trajectory
estimation accuracy and mapping quality, while achieving better storage
efficiency compared with previous online 3D reconstruction approaches. Our
implementation is available at this https URL

本篇论文提出基于概率局部隐式 Voxels (PLIVoxs) 的 DI-Fusion (Deep Implicit Fusion) 方法，通过深度神经网络将先验知识编码到 PLIVoxs 中，实现了在线 3D 重建，相较于之前的方法在存储效率和相机轨迹估计精度等方面均体现出明显优势。

DI-Fusion：基于深度先验的在线隐式三维重建

DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors

This paper focuses on visual semantic navigation, the task of producing
actions for an active agent to navigate to a specified target object category
in an unknown environment. To complete this task, the algorithm should
simultaneously locate and navigate to an instance of the category. In
comparison to the traditional point goal navigation, this task requires the
agent to have a stronger contextual prior to indoor environments. We introduce
SSCNav, an algorithm that explicitly models scene priors using a
confidence-aware semantic scene completion module to complete the scene and
guide the agent's navigation planning. Given a partial observation of the
environment, SSCNav first infers a complete scene representation with semantic
labels for the unobserved scene together with a confidence map associated with
its own prediction. Then, a policy network infers the action from the scene
completion result and confidence map. Our experiments demonstrate that the
proposed scene completion module improves the efficiency of the downstream
navigation policies. Video, code, and data: this https URL

探讨了通过使用语义场景完整模块来辅助导航规划的任务，这个模块可以同时定位和导航到所需要的物体，提高了导航策略的效率。