In order for robots to follow open-ended instructions like "go open the brown
cabinet over the sink", they require an understanding of both the scene
geometry and the semantics of their environment. Robotic systems often handle
these through separate pipelines, sometimes using very different representation
spaces, which can be suboptimal when the two objectives conflict. In this work,
we present "method", a simple method for constructing a world representation
that encodes both the semantics and spatial affordances of a scene in a
differentiable map. This allows us to build a gradient-based planner which can
navigate to locations in the scene specified using open-ended vocabulary. We
use this planner to consistently generate trajectories which are both shorter
5-10% shorter and 10-30% closer to our goal query in CLIP embedding space than
paths from comparable grid-based planners which don't leverage gradient
information. To our knowledge, this is the first end-to-end differentiable
planner optimizes for both semantics and affordance in a single implicit map.
Code and visuals are available at our website: this https URL

本研究提出了一种简单的方法来构造一个可微分的世界表示，该表示将场景的语意和空间可负载特征编码成一个映射表，从而实现了同时优化场景语意和可负载特征的端到端可微规划。

USA-Net: 机器人记忆的统一语义和可负担表示

USA-Net: Unified Semantic and Affordance Representations for Robot  Memory

We present BYOL-Explore, a conceptually simple yet general approach for
curiosity-driven exploration in visually-complex environments. BYOL-Explore
learns a world representation, the world dynamics, and an exploration policy
all-together by optimizing a single prediction loss in the latent space with no
additional auxiliary objective. We show that BYOL-Explore is effective in
DM-HARD-8, a challenging partially-observable continuous-action
hard-exploration benchmark with visually-rich 3-D environments. On this
benchmark, we solve the majority of the tasks purely through augmenting the
extrinsic reward with BYOL-Explore s intrinsic reward, whereas prior work could
only get off the ground with human demonstrations. As further evidence of the
generality of BYOL-Explore, we show that it achieves superhuman performance on
the ten hardest exploration games in Atari while having a much simpler design
than other competitive agents.

BYOL-Explore 通过优化单一预测损失，在潜在空间中学习世界表示、世界动态和探索策略，实现在视觉复杂环境下的好奇心驱动探索。此方法可有效解决部分可观察连续行动的具有挑战性的难度探索基准和 Atari 游戏等任务。