Model-based planners for partially observable problems must accommodate both
model uncertainty during planning and goal uncertainty during objective
inference. However, model-based planners may be brittle under these types of
uncertainty because they rely on an exact model and tend to commit to a single
optimal behavior. Inspired by results in the model-free setting, we propose an
entropy-regularized model-based planner for partially observable problems.
Entropy regularization promotes policy robustness for planning and objective
inference by encouraging policies to be no more committed to a single action
than necessary. We evaluate the robustness and objective inference performance
of entropy-regularized policies in three problem domains. Our results show that
entropy-regularized policies outperform non-entropy-regularized baselines in
terms of higher expected returns under modeling errors and higher accuracy
during objective inference.

模型基于策划者在部分可观测问题中必须适应策划过程中的模型和目标不确定性。受无模型设置的结果启发，我们提出了一种熵正则化的模型基于策划者来解决部分可观测问题。通过鼓励策略在规划和目标推理中不过度承诺单一行动，熵正则化提升了策略的鲁棒性和目标推理性能。我们在三个问题领域评估了熵正则化策略的鲁棒性和目标推理性能，并结果显示，在模型错误下的期望回报更高，目标推理中的准确性更高。

熵正则化的基于点的价值迭代

Entropy-regularized Point-based Value Iteration

Deformable object manipulation presents a unique set of challenges in robotic
manipulation by exhibiting high degrees of freedom and severe self-occlusion.
State representation for materials that exhibit plastic behavior, like modeling
clay or bread dough, is also difficult because they permanently deform under
stress and are constantly changing shape. In this work, we investigate each of
these challenges using the task of robotic sculpting with a parallel gripper.
We propose a system that uses point clouds as the state representation and
leverages pre-trained point cloud reconstruction Transformer to learn a latent
dynamics model to predict material deformations given a grasp action. We design
a novel action sampling algorithm that reasons about geometrical differences
between point clouds to further improve the efficiency of model-based planners.
All data and experiments are conducted entirely in the real world. Our
experiments show the proposed system is able to successfully capture the
dynamics of clay, and is able to create a variety of simple shapes.

通过使用点云作为状态表示并利用预训练的点云重建 Transformer 学习潜在动力学模型，我们提出了一种系统来解决可变形物体操纵中的挑战，进而应用于机器人雕塑任务。我们设计了一种新颖的动作采样算法，通过推理点云之间的几何差异进一步提高了基于模型的规划器的效率。所有的数据和实验都在真实世界中进行。我们的实验证明，所提出的系统能够成功地捕捉到黏土的动力学，并能够创建各种简单的形状。

SculptBot: 三维可变形物体操纵的预训练模型

SculptBot: Pre-Trained Models for 3D Deformable Object Manipulation

Domain-general model-based planners often derive their generality by
constructing search heuristics through the relaxation or abstraction of
symbolic world models. We illustrate how abstract interpretation can serve as a
unifying framework for these abstraction-based heuristics, extending the reach
of heuristic search to richer world models that make use of more complex
datatypes and functions (e.g. sets, geometry), and even models with uncertainty
and probabilistic effects. These heuristics can also be integrated with
learning, allowing agents to jumpstart planning in novel world models via
abstraction-derived information that is later refined by experience. This
suggests that abstract interpretation can play a key role in building universal
reasoning systems.

本文介绍了如何通过抽象解释作为一个统一框架来构建搜索启发式，以扩展启发式搜索到使用更复杂的数据类型和函数（例如，集合，几何），甚至具有不确定性和概率影响的更丰富的世界模型，并展示了这些启发式可与学习相结合，允许代理在新颖的世界模型中借助由抽象推导出的信息快速启动计划。这表明抽象解释在构建通用推理系统中具有关键作用。