Recent work in deep reinforcement learning (RL) has produced algorithms
capable of mastering challenging games such as Go, chess, or shogi. In these
works the RL agent directly observes the natural state of the game and controls
that state directly with its actions. However, when humans play such games,
they do not just reason about the moves but also interact with their physical
environment. They understand the state of the game by looking at the physical
board in front of them and modify it by manipulating pieces using touch and
fine-grained motor control. Mastering complicated physical systems with
abstract goals is a central challenge for artificial intelligence, but it
remains out of reach for existing RL algorithms. To encourage progress towards
this goal we introduce a set of physically embedded planning problems and make
them publicly available. We embed challenging symbolic tasks (Sokoban,
tic-tac-toe, and Go) in a physics engine to produce a set of tasks that require
perception, reasoning, and motor control over long time horizons. Although
existing RL algorithms can tackle the symbolic versions of these tasks, we find
that they struggle to master even the simplest of their physically embedded
counterparts. As a first step towards characterizing the space of solution to
these tasks, we introduce a strong baseline that uses a pre-trained expert game
player to provide hints in the abstract space to an RL agent's policy while
training it on the full sensorimotor control task. The resulting agent solves
many of the tasks, underlining the need for methods that bridge the gap between
abstract planning and embodied control. See illustrating video at
this https URL

通过在物理引擎中嵌入具有挑战性的符号任务（Sokoban，井字棋和围棋），引出了一组需要长时间视觉、推理和电机控制的任务，研究了现有的强化学习算法在这样的物理系统中的表现。同时，研究提出了使用预训练的专家游戏玩家为强化学习提供有用提示，缩小抽象规划和物体控制之间的差距的方案。