Mobile agents that can leverage help from humans can potentially accomplish
more complex tasks than they could entirely on their own. We develop "Help,
Anna!" (HANNA), an interactive photo-realistic simulator in which an agent
fulfills object-finding tasks by requesting and interpreting natural
language-and-vision assistance. An agent solving tasks in a HANNA environment
can leverage simulated human assistants, called ANNA (Automatic Natural
Navigation Assistants), which, upon request, provide natural language and
visual instructions to direct the agent towards the goals. To address the HANNA
problem, we develop a memory-augmented neural agent that hierarchically models
multiple levels of decision-making, and an imitation learning algorithm that
teaches the agent to avoid repeating past mistakes while simultaneously
predicting its own chances of making future progress. Empirically, our approach
is able to ask for help more effectively than competitive baselines and, thus,
attains higher task success rate on both previously seen and previously unseen
environments. We publicly release code and data at
this https URL . A video demo is available at
this https URL .

本研究开发了一个交互式真实仿真平台 “Help, Anna!”（HANNA），其中一个代理可通过请求和解释自然语言和视觉援助来完成物体寻找任务，解决 HANNA 问题，我们开发了一种记忆增强的神经代理和一种以模仿学习为基础的算法，通过有效地询问帮助，并同时预测自己未来的进展的机会，来比其他竞争基线更有效地避免重复错误，从而在以前看到和以前没看到的环境中获得更高的任务成功率。