General-purpose agents require fine-grained controls and rich sensory inputs
to perform a wide range of tasks. However, this complexity often leads to
intractable decision-making. Traditionally, agents are provided with
task-specific action and observation spaces to mitigate this challenge, but
this reduces autonomy. Instead, agents must be capable of building state-action
spaces at the correct abstraction level from their sensorimotor experiences. We
leverage the structure of a given set of temporally-extended actions to learn
abstract Markov decision processes (MDPs) that operate at a higher level of
temporal and state granularity. We characterize state abstractions necessary to
ensure that planning with these skills, by simulating trajectories in the
abstract MDP, results in policies with bounded value loss in the original MDP.
We evaluate our approach in goal-based navigation environments that require
continuous abstract states to plan successfully and show that abstract model
learning improves the sample efficiency of planning and learning.

通过学习抽象 MDP（Markov 决策过程）来提高智能体在多任务环境中的决策和学习效率。