Robots need to explore their surroundings to adapt to and tackle tasks in
unknown environments. Prior work has proposed building scene graphs of the
environment but typically assumes that the environment is static, omitting
regions that require active interactions. This severely limits their ability to
handle more complex tasks in household and office environments: before setting
up a table, robots must explore drawers and cabinets to locate all utensils and
condiments. In this work, we introduce the novel task of interactive scene
exploration, wherein robots autonomously explore environments and produce an
action-conditioned scene graph (ACSG) that captures the structure of the
underlying environment. The ACSG accounts for both low-level information, such
as geometry and semantics, and high-level information, such as the
action-conditioned relationships between different entities in the scene. To
this end, we present the Robotic Exploration (RoboEXP) system, which
incorporates the Large Multimodal Model (LMM) and an explicit memory design to
enhance our system's capabilities. The robot reasons about what and how to
explore an object, accumulating new information through the interaction process
and incrementally constructing the ACSG. We apply our system across various
real-world settings in a zero-shot manner, demonstrating its effectiveness in
exploring and modeling environments it has never seen before. Leveraging the
constructed ACSG, we illustrate the effectiveness and efficiency of our RoboEXP
system in facilitating a wide range of real-world manipulation tasks involving
rigid, articulated objects, nested objects like Matryoshka dolls, and
deformable objects like cloth.

通过交互式场景探索，RoboEXP 系统利用动作条件的场景图（ACSG）有效建模并推进机器人在各种真实环境中的操作任务。

RoboEXP：基于交互式探索的机器人操作行为条件化场景图

RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for  Robotic Manipulation

Solving control tasks in complex environments automatically through learning
offers great potential. While contemporary techniques from deep reinforcement
learning (DRL) provide effective solutions, their decision-making is not
transparent. We aim to provide insights into the decisions faced by the agent
by learning an automaton model of environmental behavior under the control of
an agent. However, for most control problems, automata learning is not scalable
enough to learn a useful model. In this work, we raise the capabilities of
automata learning such that it is possible to learn models for environments
that have complex and continuous dynamics.
The core of the scalability of our method lies in the computation of an
abstract state-space representation, by applying dimensionality reduction and
clustering on the observed environmental state space. The stochastic
transitions are learned via passive automata learning from observed
interactions of the agent and the environment. In an iterative model-based RL
process, we sample additional trajectories to learn an accurate environment
model in the form of a discrete-state Markov decision process (MDP). We apply
our automata learning framework on popular RL benchmarking environments in the
OpenAI Gym, including LunarLander, CartPole, Mountain Car, and Acrobot. Our
results show that the learned models are so precise that they enable the
computation of policies solving the respective control tasks. Yet the models
are more concise and more general than neural-network-based policies and by
using MDPs we benefit from a wealth of tools available for analyzing them. When
solving the task of LunarLander, the learned model even achieved similar or
higher rewards than deep RL policies learned with stable-baselines3.

通过深度强化学习、自动化学习和马尔可夫决策过程等技术，学习出由自主智能体控制的环境模型，以解决复杂环境下的控制问题，并在多个强化学习基准环境中验证了方法的有效性。

连续随机动力学学习环境模型

Learning Environment Models with Continuous Stochastic Dynamics

Autonomous agents acting in real-world environments often need to reason with
unknown novelties interfering with their plan execution. Novelty is an
unexpected phenomenon that can alter the core characteristics, composition, and
dynamics of the environment. Novelty can occur at any time in any sufficiently
complex environment without any prior notice or explanation. Previous studies
show that novelty has catastrophic impact on agent performance. Intelligent
agents reason with an internal model of the world to understand the intricacies
of their environment and to successfully execute their plans. The introduction
of novelty into the environment usually renders their internal model inaccurate
and the generated plans no longer applicable. Novelty is particularly prevalent
in the real world where domain-specific and even predicted novelty-specific
approaches are used to mitigate the novelty's impact. In this work, we
demonstrate that a domain-independent AI agent designed to detect,
characterize, and accommodate novelty in smaller-scope physics-based games such
as Angry Birds and Cartpole can be adapted to successfully perform and reason
with novelty in realistic high-fidelity simulator of the military domain.

本文旨在探索如何让自主智能系统在真实世界的环境中适应、解决并执行计划，针对其中出现的未知问题和新的情况需要进行建模和检测。