Learned visual dynamics models have proven effective for robotic manipulation
tasks. Yet, it remains unclear how best to represent scenes involving
multi-object interactions. Current methods decompose a scene into discrete
objects, but they struggle with precise modeling and manipulation amid
challenging lighting conditions as they only encode appearance tied with
specific illuminations. In this work, we propose using object-centric neural
scattering functions (OSFs) as object representations in a model-predictive
control framework. OSFs model per-object light transport, enabling
compositional scene re-rendering under object rearrangement and varying
lighting conditions. By combining this approach with inverse parameter
estimation and graph-based neural dynamics models, we demonstrate improved
model-predictive control performance and generalization in compositional
multi-object environments, even in previously unseen scenarios and harsh
lighting conditions.

本文介绍一种将基于物体中心的神经散射函数作为模型预测控制框架中的物体表示的方法，在组成多物体环境下不断变化的光照条件下，通过结合反参数估计和基于图形的神经动力学模型的方法，大大提高了模型的控制性能和推广能力。

基于对象中心神经散射函数的多对象操作

Multi-Object Manipulation via Object-Centric Neural Scattering Functions

We present a method for composing photorealistic scenes from captured images
of objects. Our work builds upon neural radiance fields (NeRFs), which
implicitly model the volumetric density and directionally-emitted radiance of a
scene. While NeRFs synthesize realistic pictures, they only model static scenes
and are closely tied to specific imaging conditions. This property makes NeRFs
hard to generalize to new scenarios, including new lighting or new arrangements
of objects. Instead of learning a scene radiance field as a NeRF does, we
propose to learn object-centric neural scattering functions (OSFs), a
representation that models per-object light transport implicitly using a
lighting- and view-dependent neural network. This enables rendering scenes even
when objects or lights move, without retraining. Combined with a volumetric
path tracing procedure, our framework is capable of rendering both intra- and
inter-object light transport effects including occlusions, specularities,
shadows, and indirect illumination. We evaluate our approach on scene
composition and show that it generalizes to novel illumination conditions,
producing photorealistic, physically accurate renderings of multi-object
scenes.

提出一种基于神经辐射场和基于对象的神经散射函数的方法，能够在不重新训练的情况下，对静态及动态场景进行光线追踪模拟，拥有在新光照和新物品排列情况下泛化能力的物理准确的多物体场景组成渲染技术。