Neural Radiance Fields (NeRFs) have shown great potential in modelling 3D
scenes. Dynamic NeRFs extend this model by capturing time-varying elements,
typically using deformation fields. The existing dynamic NeRFs employ a similar
Eulerian representation for both light radiance and deformation fields. This
leads to a close coupling of appearance and motion and lacks a physical
interpretation. In this work, we propose Dynamic Appearance Particle Neural
Radiance Field (DAP-NeRF), which introduces particle-based representation to
model the motions of visual elements in a dynamic 3D scene. DAP-NeRF consists
of superposition of a static field and a dynamic field. The dynamic field is
quantised as a collection of {\em appearance particles}, which carries the
visual information of a small dynamic element in the scene and is equipped with
a motion model. All components, including the static field, the visual features
and motion models of the particles, are learned from monocular videos without
any prior geometric knowledge of the scene. We develop an efficient
computational framework for the particle-based model. We also construct a new
dataset to evaluate motion modelling. Experimental results show that DAP-NeRF
is an effective technique to capture not only the appearance but also the
physically meaningful motions in a 3D dynamic scene.

提出了基于粒子表示的动态外观粒子神经辐射场（DAP-NeRF）模型，用于捕捉 3D 动态场景中的外观和运动，通过学习单目视频构建了有效的计算框架和新的数据集来评估运动建模效果。

动态外观粒子神经辐射场

Dynamic Appearance Particle Neural Radiance Field

Modeling and manipulating elasto-plastic objects are essential capabilities
for robots to perform complex industrial and household interaction tasks (e.g.,
stuffing dumplings, rolling sushi, and making pottery). However, due to the
high degree of freedom of elasto-plastic objects, significant challenges exist
in virtually every aspect of the robotic manipulation pipeline, e.g.,
representing the states, modeling the dynamics, and synthesizing the control
signals. We propose to tackle these challenges by employing a particle-based
representation for elasto-plastic objects in a model-based planning framework.
Our system, RoboCraft, only assumes access to raw RGBD visual observations. It
transforms the sensing data into particles and learns a particle-based dynamics
model using graph neural networks (GNNs) to capture the structure of the
underlying system. The learned model can then be coupled with model-predictive
control (MPC) algorithms to plan the robot's behavior. We show through
experiments that with just 10 minutes of real-world robotic interaction data,
our robot can learn a dynamics model that can be used to synthesize control
signals to deform elasto-plastic objects into various target shapes, including
shapes that the robot has never encountered before. We perform systematic
evaluations in both simulation and the real world to demonstrate the robot's
manipulation capabilities and ability to generalize to a more complex action
space, different tool shapes, and a mixture of motion modes. We also conduct
comparisons between RoboCraft and untrained human subjects controlling the
gripper to manipulate deformable objects in both simulation and the real world.
Our learned model-based planning framework is comparable to and sometimes
better than human subjects on the tested tasks.

RoboCraft 通过使用基于粒子的表示法，结合模型规划框架、图神经网络和模型预测控制算法，可以在只有 RGBD 视觉观测数据的情况下，学习弹塑性物体的动力学模型，并用于控制机器人进行物体变形任务，实验结果表明 RoboCraft 的优异性能在一些任务上甚至好于未训练的人类.

RoboCraft: 用图形网络学习看、模拟和塑造弹性塑性物体

RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks

Humans intuitively recognize objects' physical properties and predict their
motion, even when the objects are engaged in complicated interactions. The
abilities to perform physical reasoning and to adapt to new environments, while
intrinsic to humans, remain challenging to state-of-the-art computational
models. In this work, we present a neural model that simultaneously reasons
about physics and makes future predictions based on visual and dynamics priors.
The visual prior predicts a particle-based representation of the system from
visual observations. An inference module operates on those particles,
predicting and refining estimates of particle locations, object states, and
physical parameters, subject to the constraints imposed by the dynamics prior,
which we refer to as visual grounding. We demonstrate the effectiveness of our
method in environments involving rigid objects, deformable materials, and
fluids. Experiments show that our model can infer the physical properties
within a few observations, which allows the model to quickly adapt to unseen
scenarios and make accurate predictions into the future.

该论文提出了一种神经模型，同时基于物理学原理和视觉先验进行推理和预测，通过对粒子的估计预测物理参数，即视觉基础，并在刚体、可变形材料和流体的复杂环境中证明了它的有效性，允许模型在几次观察内推断出物理特性，从而快速适应未见过的场景并进行准确的未来预测。