Task and Motion Planning (TAMP) approaches are effective at planning
long-horizon autonomous robot manipulation. However, because they require a
planning model, it can be difficult to apply them to domains where the
environment and its dynamics are not fully known. We propose to overcome these
limitations by leveraging deep generative modeling, specifically diffusion
models, to learn constraints and samplers that capture these
difficult-to-engineer aspects of the planning model. These learned samplers are
composed and combined within a TAMP solver in order to find action parameter
values jointly that satisfy the constraints along a plan. To tractably make
predictions for unseen objects in the environment, we define these samplers on
low-dimensional learned latent embeddings of changing object state. We evaluate
our approach in an articulated object manipulation domain and show how the
combination of classical TAMP, generative learning, and latent embeddings
enables long-horizon constraint-based reasoning.

本文介绍了将深度生成建模与任务规划相结合的方法，以解决在未知环境中的自主机器人操作规划中运用传统 TAMP 方法中存在的问题。通过使用扩散模型学习约束条件和采样器，并将其与 TAMP 求解器组合使用以满足计划中的约束条件。同时，为了在这些约束条件满足不确定物体状态的情况下进行预测，在学习的低维潜在空间中定义这些采样器。实验结果中介绍了使用传统 TAMP，生成式学习和潜在嵌入组合的长程约束条件推理方法在一个物体操作环境中的有效性。

DiMSam：部分可观察性下任务与动作规划的扩散模型采样器

DiMSam: Diffusion Models as Samplers for Task and Motion Planning under  Partial Observability

Deep imitation learning is a promising approach that does not require
hard-coded control rules in autonomous robot manipulation. The current
applications of deep imitation learning to robot manipulation have been limited
to reactive control based on the states at the current time step. However,
future robots will also be required to solve tasks utilizing their memory
obtained by experience in complicated environments (e.g., when the robot is
asked to find a previously used object on a shelf). In such a situation, simple
deep imitation learning may fail because of distractions caused by complicated
environments. We propose that gaze prediction from sequential visual input
enables the robot to perform a manipulation task that requires memory. The
proposed algorithm uses a Transformer-based self-attention architecture for the
gaze estimation based on sequential data to implement memory. The proposed
method was evaluated with a real robot multi-object manipulation task that
requires memory of the previous states.

本文提出了一种使用基于 Transformer 的自注意力结构实现眼神预测，以实现具有记忆的机器人操作任务的算法。通过将机器人视觉输入的连续序列作为输入，该算法能够通过使用传统的深度模仿学习方法并结合眼动预测来实现多物体操作任务。实验结果表明，该方法成功地改善了机器人在具有记忆要求的复杂环境中的操作表现。