Temporal abstraction and efficient planning pose significant challenges in
offline reinforcement learning, mainly when dealing with domains that involve
temporally extended tasks and delayed sparse rewards. Existing methods
typically plan in the raw action space and can be inefficient and inflexible.
Latent action spaces offer a more flexible paradigm, capturing only possible
actions within the behavior policy support and decoupling the temporal
structure between planning and modeling. However, current latent-action-based
methods are limited to discrete spaces and require expensive planning. This
paper presents a unified framework for continuous latent action space
representation learning and planning by leveraging latent, score-based
diffusion models. We establish the theoretical equivalence between planning in
the latent action space and energy-guided sampling with a pretrained diffusion
model and incorporate a novel sequence-level exact sampling method. Our
proposed method, $\texttt{LatentDiffuser}$, demonstrates competitive
performance on low-dimensional locomotion control tasks and surpasses existing
methods in higher-dimensional tasks.

该研究论文介绍了一种统一的框架，利用潜在的、基于分数的扩散模型进行连续潜在行动空间的表示学习和规划，通过考虑预训练的扩散模型进行能量引导抽样，通过引入一种新颖的序列级别的精确抽样方法。所提出的方法在低维机动控制任务上表现出竞争性能，并在高维任务中超过了现有的方法。

潜在扩散下的高效规划

Efficient Planning with Latent Diffusion

Efficient planning in continuous state and action spaces is fundamentally
hard, even when the transition model is deterministic and known. One way to
alleviate this challenge is to perform bilevel planning with abstractions,
where a high-level search for abstract plans is used to guide planning in the
original transition space. Previous work has shown that when state abstractions
in the form of symbolic predicates are hand-designed, operators and samplers
for bilevel planning can be learned from demonstrations. In this work, we
propose an algorithm for learning predicates from demonstrations, eliminating
the need for manually specified state abstractions. Our key idea is to learn
predicates by optimizing a surrogate objective that is tractable but faithful
to our real efficient-planning objective. We use this surrogate objective in a
hill-climbing search over predicate sets drawn from a grammar. Experimentally,
we show across four robotic planning environments that our learned abstractions
are able to quickly solve held-out tasks, outperforming six baselines. Code:
this https URL

本文提出了一种从示例中学习抽象谓词的算法，通过优化可行但准确的替代目标，通过一组语法中提取的谓词集合进行爬山搜索，实验结果表明，我们的学习抽象能够快速解决悬而未决的任务，并优于六种基线。

双层规划的谓词发现

Predicate Invention for Bilevel Planning

In partially observed environments, it can be useful for a human to provide
the robot with declarative information that represents probabilistic relational
constraints on properties of objects in the world, augmenting the robot's
sensory observations. For instance, a robot tasked with a search-and-rescue
mission may be informed by the human that two victims are probably in the same
room. An important question arises: how should we represent the robot's
internal knowledge so that this information is correctly processed and combined
with raw sensory information? In this paper, we provide an efficient belief
state representation that dynamically selects an appropriate factoring,
combining aspects of the belief when they are correlated through information
and separating them when they are not. This strategy works in open domains, in
which the set of possible objects is not known in advance, and provides
significant improvements in inference time over a static factoring, leading to
more efficient planning for complex partially observed tasks. We validate our
approach experimentally in two open-domain planning problems: a 2D discrete
gridworld task and a 3D continuous cooking task. A supplementary video can be
found at this http URL

该篇论文介绍了一个关于机器人在部分观察环境下进行可靠的推理与规划的有效方法，通过提供现实世界中物体属性的概率信息，并采取正确的策略，实现基于内部状态的自我学习，并验证了该方法在两个开放领域的规划问题中的有效性。