The domain of Embodied AI has recently witnessed substantial progress, particularly in navigating agents within their environments. These early successes have laid the building blocks for the community to tackle tasks that require agents to actively interact with objects in their environment. Object manipulation is an established research domain within the robotics community and poses several challenges including manipulator motion, grasping and long-horizon planning, particularly when dealing with oft-overlooked practical setups involving visually rich and complex scenes, manipulation using mobile agents (as opposed to tabletop manipulation), and generalization to unseen environments and objects. We propose a framework for object manipulation built upon the physics-enabled, visually rich AI2-THOR framework and present a new challenge to the Embodied AI community known as ArmPointNav. This task extends the popular point navigation task to object manipulation and offers new challenges including 3D obstacle avoidance, manipulating objects in the presence of occlusion, and multi-object manipulation that necessitates long term planning. Popular learning paradigms that are successful on PointNav challenges show promise, but leave a large room for improvement.

本文提出了一个基于 AI2-THOR 框架的物体操作框架并提出了一个名为 ArmPointNav 的新挑战任务，以扩展点导航任务到物体操作，并提供了新的挑战，包括 3D 障碍物避免、存在遮挡的物体操作以及多物体操作，并指出了 PointNav 挑战中成功的流行学习范例仍有大量改进空间。

ManipulaTHOR：一个用于视觉物体操纵的框架