Passive visual systems typically fail to recognize objects in the amodal
setting where they are heavily occluded. In contrast, humans and other embodied
agents have the ability to move in the environment, and actively control the
viewing angle to better understand object shapes and semantics. In this work,
we introduce the task of Embodied Visual Recognition (EVR): An agent is
instantiated in a 3D environment close to an occluded target object, and is
free to move in the environment to perform object classification, amodal object
localization, and amodal object segmentation. To address this, we develop a new
model called Embodied Mask R-CNN, for agents to learn to move strategically to
improve their visual recognition abilities. We conduct experiments using the
House3D environment. Experimental results show that: 1) agents with embodiment
(movement) achieve better visual recognition performance than passive ones; 2)
in order to improve visual recognition abilities, agents can learn strategical
moving paths that are different from shortest paths.

本文提出了一种名为 “Embodied Visual Recognition” 的任务，其中代理可以在三维环境中移动以对被遮挡的目标物体进行识别、定位和分割，研究结果表明，具有移动能力的代理比被动代理的视觉识别性能更好，代理可以学习不同于最短路径的策略移动路径来提高视觉识别能力。