Performing language-conditioned robotic manipulation tasks in unstructured
environments is highly demanded for general intelligent robots. Conventional
robotic manipulation methods usually learn semantic representation of the
observation for action prediction, which ignores the scene-level spatiotemporal
dynamics for human goal completion. In this paper, we propose a dynamic
Gaussian Splatting method named ManiGaussian for multi-task robotic
manipulation, which mines scene dynamics via future scene reconstruction.
Specifically, we first formulate the dynamic Gaussian Splatting framework that
infers the semantics propagation in the Gaussian embedding space, where the
semantic representation is leveraged to predict the optimal robot action. Then,
we build a Gaussian world model to parameterize the distribution in our dynamic
Gaussian Splatting framework, which provides informative supervision in the
interactive environment via future scene reconstruction. We evaluate our
ManiGaussian on 10 RLBench tasks with 166 variations, and the results
demonstrate our framework can outperform the state-of-the-art methods by 13.1\%
in average success rate.

在这篇论文中，我们提出了一种名为 ManiGaussian 的动态高斯斑点方法，用于多任务机器人操作，该方法通过未来场景重建来挖掘场景动态性。我们首先制定了动态高斯斑点框架，推断高斯嵌入空间中的语义传播，利用语义表示来预测最佳机器人动作。然后，我们建立了一个高斯世界模型来参数化我们的动态高斯斑点框架中的分布，通过未来场景重建在交互环境中提供信息化监督。我们在 10 个 RLBench 任务的 166 个变体上评估了 ManiGaussian，并且结果表明我们的框架在平均成功率上比现有最先进方法提高了 13.1%。

ManiGaussian: 多任务机器人操作的动态高斯溅射

ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic  Manipulation

This paper introduces Hierarchical Diffusion Policy (HDP), a hierarchical
agent for multi-task robotic manipulation. HDP factorises a manipulation policy
into a hierarchical structure: a high-level task-planning agent which predicts
a distant next-best end-effector pose (NBP), and a low-level goal-conditioned
diffusion policy which generates optimal motion trajectories. The factorised
policy representation allows HDP to tackle both long-horizon task planning
while generating fine-grained low-level actions. To generate context-aware
motion trajectories while satisfying robot kinematics constraints, we present a
novel kinematics-aware goal-conditioned control agent, Robot Kinematics
Diffuser (RK-Diffuser). Specifically, RK-Diffuser learns to generate both the
end-effector pose and joint position trajectories, and distill the accurate but
kinematics-unaware end-effector pose diffuser to the kinematics-aware but less
accurate joint position diffuser via differentiable kinematics. Empirically, we
show that HDP achieves a significantly higher success rate than the
state-of-the-art methods in both simulation and real-world.

介绍了一种名为 Hierarchical Diffusion Policy (HDP) 的多任务机器人操作策略，包括分层结构、任务规划、运动轨迹和运动规划中的运动学感知控制。实验证明，HDP 在模拟和实际环境中的成功率显著高于现有方法。