Large-scale endeavors like RT-1 and widespread community efforts such as
Open-X-Embodiment have contributed to growing the scale of robot demonstration
data. However, there is still an opportunity to improve the quality, quantity,
and diversity of robot demonstration data. Although vision-language models have
been shown to automatically generate demonstration data, their utility has been
limited to environments with privileged state information, they require
hand-designed skills, and are limited to interactions with few object
instances. We propose Manipulate-Anything, a scalable automated generation
method for real-world robotic manipulation. Unlike prior work, our method can
operate in real-world environments without any privileged state information,
hand-designed skills, and can manipulate any static object. We evaluate our
method using two setups. First, Manipulate-Anything successfully generates
trajectories for all 5 real-world and 12 simulation tasks, significantly
outperforming existing methods like VoxPoser. Second, Manipulate-Anything's
demonstrations can train more robust behavior cloning policies than training
with human demonstrations, or from data generated by VoxPoser and
Code-As-Policies. We believe \methodLong\ can be the scalable method for both
generating data for robotics and solving novel tasks in a zero-shot setting.

Manipulate-Anything 是一种可扩展的自动化生成方法，在没有特权状态信息、手动设计的技能的情况下，在真实环境中操作任何静态物体，其生成的演示数据可以用于生成鲁棒的行为克隆策略。

自动化现实世界机器人：利用视觉语言模型操控一切

Manipulate-Anything: Automating Real-World Robots using Vision-Language  Models

With the rise of stochastic generative models in robot policy learning,
end-to-end visuomotor policies are increasingly successful at solving complex
tasks by learning from human demonstrations. Nevertheless, since real-world
evaluation costs afford users only a small number of policy rollouts, it
remains a challenge to accurately gauge the performance of such policies. This
is exacerbated by distribution shifts causing unpredictable changes in
performance during deployment. To rigorously evaluate behavior cloning
policies, we present a framework that provides a tight lower-bound on robot
performance in an arbitrary environment, using a minimal number of experimental
policy rollouts. Notably, by applying the standard stochastic ordering to robot
performance distributions, we provide a worst-case bound on the entire
distribution of performance (via bounds on the cumulative distribution
function) for a given task. We build upon established statistical results to
ensure that the bounds hold with a user-specified confidence level and
tightness, and are constructed from as few policy rollouts as possible. In
experiments we evaluate policies for visuomotor manipulation in both simulation
and hardware. Specifically, we (i) empirically validate the guarantees of the
bounds in simulated manipulation settings, (ii) find the degree to which a
learned policy deployed on hardware generalizes to new real-world environments,
and (iii) rigorously compare two policies tested in out-of-distribution
settings. Our experimental data, code, and implementation of confidence bounds
are open-source.

通过一小部分策略演练，我们提出了一个为机器人在任意环境中的性能提供严格下界评估的框架，通过应用标准的随机排序来提供性能分布的最差情况边界，并确保边界在用户指定的置信水平和紧凑度上成立。