Neural fields have recently enjoyed great success in representing and
rendering 3D scenes. However, most state-of-the-art implicit representations
model static or dynamic scenes as a whole, with minor variations. Existing work
on learning disentangled world and object neural fields do not consider the
problem of composing objects into different world neural fields in a
lighting-aware manner. We present Lighting-Aware Neural Field (LANe) for the
compositional synthesis of driving scenes in a physically consistent manner.
Specifically, we learn a scene representation that disentangles the static
background and transient elements into a world-NeRF and class-specific
object-NeRFs to allow compositional synthesis of multiple objects in the scene.
Furthermore, we explicitly designed both the world and object models to handle
lighting variation, which allows us to compose objects into scenes with
spatially varying lighting. This is achieved by constructing a light field of
the scene and using it in conjunction with a learned shader to modulate the
appearance of the object NeRFs. We demonstrate the performance of our model on
a synthetic dataset of diverse lighting conditions rendered with the CARLA
simulator, as well as a novel real-world dataset of cars collected at different
times of the day. Our approach shows that it outperforms state-of-the-art
compositional scene synthesis on the challenging dataset setup, via composing
object-NeRFs learned from one scene into an entirely different scene whilst
still respecting the lighting variations in the novel scene. For more results,
please visit our project website this https URL

本文提出了一种名为 Lighting-Aware Neural Field (LANe) 的方法，以在具有物理一致性的条件下进行组合合成物体，该方法采用神经场来表征 3D 场景，将静态背景和瞬态元素分离成世界 - NeRF 和类特定的物体 - NeRFs，并明确地设计了世界和物体模型以处理照明变化，以便将物体组合到具有空间变化照明的场景中。

LANe: 用于组合场景合成的光照感知神经场

LANe: Lighting-Aware Neural Fields for Compositional Scene Synthesis

Modelling individual objects in a scene as Neural Radiance Fields (NeRFs)
provides an alternative geometric scene representation that may benefit
downstream robotics tasks such as scene understanding and object manipulation.
However, we identify three challenges to using real-world training data
collected by a robot to train a NeRF: (i) The camera trajectories are
constrained, and full visual coverage is not guaranteed - especially when
obstructions to the objects of interest are present; (ii) the poses associated
with the images are noisy due to odometry or localization noise; (iii) the
objects are not easily isolated from the background. This paper evaluates the
extent to which above factors degrade the quality of the learnt implicit object
representation. We introduce a pipeline that decomposes a scene into multiple
individual object-NeRFs, using noisy object instance masks and bounding boxes,
and evaluate the sensitivity of this pipeline with respect to noisy poses,
instance masks, and the number of training images. We uncover that the
sensitivity to noisy instance masks can be partially alleviated with depth
supervision and quantify the importance of including the camera extrinsics in
the NeRF optimisation process.

本文评估了使用受限的真实世界数据来训练神经辐射场中的单个对象表示的质量，并介绍了一种将场景分解成多个基于对象的 NeRF 的流水线，使用嘈杂的对象实例掩码和边界框，并评估了该流水线对噪声位姿、实例掩码和训练图像数量的敏感性。