Large-scale endeavors like RT-1 and widespread community efforts such as
Open-X-Embodiment have contributed to growing the scale of robot demonstration
data. However, there is still an opportunity to improve the quality, quantity,
and diversity of robot demonstration data. Although vision-language models have
been shown to automatically generate demonstration data, their utility has been
limited to environments with privileged state information, they require
hand-designed skills, and are limited to interactions with few object
instances. We propose Manipulate-Anything, a scalable automated generation
method for real-world robotic manipulation. Unlike prior work, our method can
operate in real-world environments without any privileged state information,
hand-designed skills, and can manipulate any static object. We evaluate our
method using two setups. First, Manipulate-Anything successfully generates
trajectories for all 5 real-world and 12 simulation tasks, significantly
outperforming existing methods like VoxPoser. Second, Manipulate-Anything's
demonstrations can train more robust behavior cloning policies than training
with human demonstrations, or from data generated by VoxPoser and
Code-As-Policies. We believe \methodLong\ can be the scalable method for both
generating data for robotics and solving novel tasks in a zero-shot setting.

Manipulate-Anything 是一种可扩展的自动化生成方法，在没有特权状态信息、手动设计的技能的情况下，在真实环境中操作任何静态物体，其生成的演示数据可以用于生成鲁棒的行为克隆策略。

自动化现实世界机器人：利用视觉语言模型操控一切

Manipulate-Anything: Automating Real-World Robots using Vision-Language  Models

In the same way that the computer vision (CV) and natural language processing
(NLP) communities have developed self-supervised methods, reinforcement
learning (RL) can be cast as a self-supervised problem: learning to reach any
goal, without requiring human-specified rewards or labels. However, actually
building a self-supervised foundation for RL faces some important challenges.
Building on prior contrastive approaches to this RL problem, we conduct careful
ablation experiments and discover that a shallow and wide architecture,
combined with careful weight initialization and data augmentation, can
significantly boost the performance of these contrastive RL approaches on
challenging simulated benchmarks. Additionally, we demonstrate that, with these
design decisions, contrastive approaches can solve real-world robotic
manipulation tasks, with tasks being specified by a single goal image provided
after training.

通过对比强化学习问题的先前对比方法，我们发现，结合谨慎的权重初始化和数据增强，使用浅而宽的架构可以显著提高这些对比强化学习方法在挑战性的模拟基准测试中的性能，并且可以通过这些设计决策解决实际的机器人操作任务。

稳定对比强化学习：离线目标达成技术

Stabilizing Contrastive RL: Techniques for Offline Goal Reaching

Model-free deep reinforcement learning has been shown to exhibit good
performance in domains ranging from video games to simulated robotic
manipulation and locomotion. However, model-free methods are known to perform
poorly when the interaction time with the environment is limited, as is the
case for most real-world robotic tasks. In this paper, we study how maximum
entropy policies trained using soft Q-learning can be applied to real-world
robotic manipulation. The application of this method to real-world manipulation
is facilitated by two important features of soft Q-learning. First, soft
Q-learning can learn multimodal exploration strategies by learning policies
represented by expressive energy-based models. Second, we show that policies
learned with soft Q-learning can be composed to create new policies, and that
the optimality of the resulting policy can be bounded in terms of the
divergence between the composed policies. This compositionality provides an
especially valuable tool for real-world manipulation, where constructing new
policies by composing existing skills can provide a large gain in efficiency
over training from scratch. Our experimental evaluation demonstrates that soft
Q-learning is substantially more sample efficient than prior model-free deep
reinforcement learning methods, and that compositionality can be performed for
both simulated and real-world tasks.

本研究探讨软 Q-learning 方法在真实世界机器人操作中的应用，证明软 Q-learning 方法比先前的模型自由深度强化学习方法具有更高的采样效率，并且可以通过将学习到的策略组合创建新的策略，从而在真实世界机器人操作中提供高效的工具。