Scalable robot learning in the real world is limited by the cost and safety
issues of real robots. In addition, rolling out robot trajectories in the real
world can be time-consuming and labor-intensive. In this paper, we propose to
learn an interactive real-robot action simulator as an alternative. We
introduce a novel method, IRASim, which leverages the power of generative
models to generate extremely realistic videos of a robot arm that executes a
given action trajectory, starting from an initial given frame. To validate the
effectiveness of our method, we create a new benchmark, IRASim Benchmark, based
on three real-robot datasets and perform extensive experiments on the
benchmark. Results show that IRASim outperforms all the baseline methods and is
more preferable in human evaluations. We hope that IRASim can serve as an
effective and scalable approach to enhance robot learning in the real world. To
promote research for generative real-robot action simulators, we open-source
code, benchmark, and checkpoints at https: //gen-irasim.github.io.

提出了一种利用生成模型的交互式真实机器人动作模拟器，通过生成极其逼真的机器人执行给定动作轨迹的视频，验证了其在实验中的有效性，并开源了相关代码和基准测试。

IRASim：学习交互式真实机器人行为模拟器

IRASim: Learning Interactive Real-Robot Action Simulators

We introduce BridgeData V2, a large and diverse dataset of robotic
manipulation behaviors designed to facilitate research on scalable robot
learning. BridgeData V2 contains 60,096 trajectories collected across 24
environments on a publicly available low-cost robot. BridgeData V2 provides
extensive task and environment variability, leading to skills that can
generalize across environments, domains, and institutions, making the dataset a
useful resource for a broad range of researchers. Additionally, the dataset is
compatible with a wide variety of open-vocabulary, multi-task learning methods
conditioned on goal images or natural language instructions. In our
experiments, we train 6 state-of-the-art imitation learning and offline
reinforcement learning methods on our dataset, and find that they succeed on a
suite of tasks requiring varying amounts of generalization. We also demonstrate
that the performance of these methods improves with more data and higher
capacity models, and that training on a greater variety of skills leads to
improved generalization. By publicly sharing BridgeData V2 and our pre-trained
models, we aim to accelerate research in scalable robot learning methods.
Project page at this https URL

介绍了 BridgeData V2，一个庞大而多样化的机器人操作行为数据集，旨在促进可扩展机器人学习的研究。