Physical construction -- the ability to compose objects, subject to physical dynamics, in order to serve some function -- is fundamental to human intelligence. Here we introduce a suite of challenging physical construction tasks inspired by how children play with blocks, such as matching a target configuration, stacking and attaching blocks to connect objects together, and creating shelter-like structures over target objects. We then examine how a range of modern deep reinforcement learning agents fare on these challenges, and introduce several new approaches which provide superior performance. Our results show that agents which use structured representations (e.g., objects and scene graphs) and structured policies (e.g., object-centric actions) outperform those which use less structured representations, and generalize better beyond their training when asked to reason about larger scenes. Agents which use model-based planning via Monte-Carlo Tree Search also outperform strictly model-free agents in our most challenging construction problems. We conclude that approaches which combine structured representations and reasoning with powerful learning are a key path toward agents that possess rich intuitive physics, scene understanding, and planning.

本文介绍的一系列具有挑战性的物理结构任务，探究了在这些任务上如何采用一系列深度强化学习代理的处理方式，并引入了多种新方法以提高性能。结果表明，使用结构化表示和结构化策略的代理比使用较少结构的代理表现更好，并且更具有推广性。同时本文还表明，将结构化表示和推理与强大的学习相结合的方法是实现丰富的直观物理、场景理解和规划代理的关键路径。

物理建设的结构代理