In this paper, we introduce Watch-And-Help (WAH), a challenge for testing
social intelligence in agents. In WAH, an AI agent needs to help a human-like
agent perform a complex household task efficiently. To succeed, the AI agent
needs to i) understand the underlying goal of the task by watching a single
demonstration of the human-like agent performing the same task (social
perception), and ii) coordinate with the human-like agent to solve the task in
an unseen environment as fast as possible (human-AI collaboration). For this
challenge, we build VirtualHome-Social, a multi-agent household environment,
and provide a benchmark including both planning and learning based baselines.
We evaluate the performance of AI agents with the human-like agent as well as
with real humans using objective metrics and subjective user ratings.
Experimental results demonstrate that the proposed challenge and virtual
environment enable a systematic evaluation on the important aspects of machine
social intelligence at scale.

本研究介绍了一个名为 Watch-And-Help（WAH）的挑战赛，用于测试智能机器人的社交智能。研究构建了多机器人家居环境 VirtualHome-Social，并提供了基准测试，通过客观指标和主观用户评分评估了智能机器人的表现，结果证明该挑战赛和虚拟环境可在规模上对机器的社交智能的重要方面进行系统评估。