We present BEHAVIOR-1K, a comprehensive simulation benchmark for
human-centered robotics. BEHAVIOR-1K includes two components, guided and
motivated by the results of an extensive survey on "what do you want robots to
do for you?". The first is the definition of 1,000 everyday activities,
grounded in 50 scenes (houses, gardens, restaurants, offices, etc.) with more
than 9,000 objects annotated with rich physical and semantic properties. The
second is OMNIGIBSON, a novel simulation environment that supports these
activities via realistic physics simulation and rendering of rigid bodies,
deformable bodies, and liquids. Our experiments indicate that the activities in
BEHAVIOR-1K are long-horizon and dependent on complex manipulation skills, both
of which remain a challenge for even state-of-the-art robot learning solutions.
To calibrate the simulation-to-reality gap of BEHAVIOR-1K, we provide an
initial study on transferring solutions learned with a mobile manipulator in a
simulated apartment to its real-world counterpart. We hope that BEHAVIOR-1K's
human-grounded nature, diversity, and realism make it valuable for embodied AI
and robot learning research. Project website: this https URL

我们提供了 BEHAVIOR-1K，这是一个全面的人类中心机器人仿真基准。BEHAVIOR-1K 包括两个组成部分，通过 “你希望机器人为你做什么？” 的广泛调查结果进行指导和激励。第一个是定义了 1,000 种日常活动，与 50 个场景（房屋、花园、餐厅、办公室等）以及超过 9,000 个带有丰富物理和语义属性的对象相联系。第二个是 OMNIGIBSON，一种通过逼真的物理模拟和渲染刚体、可变形体和液体来支持这些活动的新型仿真环境。我们的实验表明，BEHAVIOR-1K 中的活动是长期目标，并且依赖于复杂的操作技能，这对于即使是最先进的机器人学习解决方案来说仍然是一个挑战。为了校准 BEHAVIOR-1K 的仿真与现实之间的差距，我们提供了一个初步研究，将在模拟公寓中学习的解决方案转移到实际环境中的移动操作机器人上。我们希望 BEHAVIOR-1K 的以人为中心的特性、多样性和逼真性对于具体化的 AI 和机器人学习研究是有价值的。项目网站：https:// 此 URL。

BEHAVIOR-1K: 一个以人为本、具体实施的人工智能基准，涵盖 1,000 个日常活动和逼真的模拟

BEHAVIOR-1K: A Human-Centered, Embodied AI Benchmark with 1,000 Everyday  Activities and Realistic Simulation

Prompt-based learning has emerged as a successful paradigm in natural
language processing, where a single general-purpose language model can be
instructed to perform any task specified by input prompts. Yet task
specification in robotics comes in various forms, such as imitating one-shot
demonstrations, following language instructions, and reaching visual goals.
They are often considered different tasks and tackled by specialized models.
This work shows that we can express a wide spectrum of robot manipulation tasks
with multimodal prompts, interleaving textual and visual tokens. We design a
transformer-based generalist robot agent, VIMA, that processes these prompts
and outputs motor actions autoregressively. To train and evaluate VIMA, we
develop a new simulation benchmark with thousands of procedurally-generated
tabletop tasks with multimodal prompts, 600K+ expert trajectories for imitation
learning, and four levels of evaluation protocol for systematic generalization.
VIMA achieves strong scalability in both model capacity and data size. It
outperforms prior SOTA methods in the hardest zero-shot generalization setting
by up to $2.9\times$ task success rate given the same training data. With
$10\times$ less training data, VIMA still performs $2.7\times$ better than the
top competing approach. We open-source all code, pretrained models, dataset,
and simulation benchmark at this https URL

使用多模态提示设计了一个基于转换器的通用机器人代理 (VIMA)，可以表达多种机器人操作任务，且在新颖的零激励泛化情况下优于先前的状态最优方法。