Key to rich, dexterous manipulation in the real world is the ability to
coordinate control across two hands. However, while the promise afforded by
bimanual robotic systems is immense, constructing control policies for dual arm
autonomous systems brings inherent difficulties. One such difficulty is the
high-dimensionality of the bimanual action space, which adds complexity to both
model-based and data-driven methods. We counteract this challenge by drawing
inspiration from humans to propose a novel role assignment framework: a
stabilizing arm holds an object in place to simplify the environment while an
acting arm executes the task. We instantiate this framework with BimanUal
Dexterity from Stabilization (BUDS), which uses a learned restabilizing
classifier to alternate between updating a learned stabilization position to
keep the environment unchanged, and accomplishing the task with an acting
policy learned from demonstrations. We evaluate BUDS on four bimanual tasks of
varying complexities on real-world robots, such as zipping jackets and cutting
vegetables. Given only 20 demonstrations, BUDS achieves 76.9% task success
across our task suite, and generalizes to out-of-distribution objects within a
class with a 52.7% success rate. BUDS is 56.0% more successful than an
unstructured baseline that instead learns a BC stabilizing policy due to the
precision required of these complex tasks. Supplementary material and videos
can be found at this https URL .

借鉴人类，提出了一种新颖的角色分配框架：一个稳定的手臂将物体固定在原位以简化环境，而另一个执行手臂执行任务。通过 BimanUal Dexterity from Stabilization (BUDS) 实现了该框架，并在真实世界的机器人上评估了其性能，在各种复杂性的四个双手任务中，仅通过 20 次演示，BUDS 在任务套件中实现了 76.9% 的成功率，且能够以 52.7% 的成功率推广到同一类别的不同对象。BUDS 比基于 BC 稳定策略的非结构化基线成功率提高了 56.0%，这是由于这些复杂任务所要求的高精度。

稳定以行动：学习协调的双手操作

Stabilize to Act: Learning to Coordinate for Bimanual Manipulation

Reinforcement learning is a promising approach to developing hard-to-engineer
adaptive solutions for complex and diverse robotic tasks. However, learning
with real-world robots is often unreliable and difficult, which resulted in
their low adoption in reinforcement learning research. This difficulty is
worsened by the lack of guidelines for setting up learning tasks with robots.
In this work, we develop a learning task with a UR5 robotic arm to bring to
light some key elements of a task setup and study their contributions to the
challenges with robots. We find that learning performance can be highly
sensitive to the setup, and thus oversights and omissions in setup details can
make effective learning, reproducibility, and fair comparison hard. Our study
suggests some mitigating steps to help future experimenters avoid difficulties
and pitfalls. We show that highly reliable and repeatable experiments can be
performed in our setup, indicating the possibility of reinforcement learning
research extensively based on real-world robots.

本篇论文研究了基于实际机器人的强化学习，提出了一种学习任务并分析了实验设置对学习性能、可重复性和公平比较的影响，并为未来的实验者提供了一些缓解措施。

利用真实世界机器人设置强化学习任务

Setting up a Reinforcement Learning Task with a Real-World Robot

Real-world robots are becoming increasingly complex and commonly act in
poorly understood environments where it is extremely challenging to model or
learn their true dynamics. Therefore, it might be desirable to take a
task-specific approach, wherein the focus is on explicitly learning the
dynamics model which achieves the best control performance for the task at
hand, rather than learning the true dynamics. In this work, we use Bayesian
optimization in an active learning framework where a locally linear dynamics
model is learned with the intent of maximizing the control performance, and
used in conjunction with optimal control schemes to efficiently design a
controller for a given task. This model is updated directly based on the
performance observed in experiments on the physical system in an iterative
manner until a desired performance is achieved. We demonstrate the efficacy of
the proposed approach through simulations and real experiments on a quadrotor
testbed.

通过基于贝叶斯优化的主动学习框架以及最优控制方案，采用特定任务方法，使用局部线性动态模型来设计给定任务的控制器，并通过对物理系统的实验来迭代地更新模型，最终在四旋翼实验台上进行仿真和实验，验证了该方法的有效性。