Automating dexterous, contact-rich manipulation tasks using rigid robots is a
significant challenge in robotics. Rigid robots, defined by their actuation
through position commands, face issues of excessive contact forces due to their
inability to adapt to contact with the environment, potentially causing damage.
While compliance control schemes have been introduced to mitigate these issues
by controlling forces via external sensors, they are hampered by the need for
fine-tuning task-specific controller parameters. Learning from Demonstrations
(LfD) offers an intuitive alternative, allowing robots to learn manipulations
through observed actions. In this work, we introduce a novel system to enhance
the teaching of dexterous, contact-rich manipulations to rigid robots. Our
system is twofold: firstly, it incorporates a teleoperation interface utilizing
Virtual Reality (VR) controllers, designed to provide an intuitive and
cost-effective method for task demonstration with haptic feedback. Secondly, we
present Comp-ACT (Compliance Control via Action Chunking with Transformers), a
method that leverages the demonstrations to learn variable compliance control
from a few demonstrations. Our methods have been validated across various
complex contact-rich manipulation tasks using single-arm and bimanual robot
setups in simulated and real-world environments, demonstrating the
effectiveness of our system in teaching robots dexterous manipulations with
enhanced adaptability and safety.

通过虚拟现实控制器提供直观且经济的任务演示方法，结合动作块划分和变量合规性控制的方法，提高了刚性机器人的灵活适应性和安全性。

从少量示范学习双臂机器人带有触觉反馈遥操作系统的可变柔顺控制

Learning Variable Compliance Control From a Few Demonstrations for  Bimanual Robot with Haptic Feedback Teleoperation System

Executing contact-rich manipulation tasks necessitates the fusion of tactile
and visual feedback. However, the distinct nature of these modalities poses
significant challenges. In this paper, we introduce a system that leverages
visual and tactile sensory inputs to enable dexterous in-hand manipulation.
Specifically, we propose Robot Synesthesia, a novel point cloud-based tactile
representation inspired by human tactile-visual synesthesia. This approach
allows for the simultaneous and seamless integration of both sensory inputs,
offering richer spatial information and facilitating better reasoning about
robot actions. The method, trained in a simulated environment and then deployed
to a real robot, is applicable to various in-hand object rotation tasks.
Comprehensive ablations are performed on how the integration of vision and
touch can improve reinforcement learning and Sim2Real performance. Our project
page is available at this https URL .

使用视觉和触觉感知输入实现灵巧的手中操作是一项具有挑战性的任务，本文提出了一种基于点云的触觉表示方法 Robot Synesthesia，该方法通过同时无缝地整合视觉和触觉输入，提供更丰富的空间信息，有助于更好地推理机器人动作，通过在模拟环境中训练并应用于真实机器人，可适用于各种手中物体旋转任务，并通过综合消融实验验证了视觉和触觉的整合如何改善强化学习和实验到真实场景的性能。

机器人共感：视触感知下的手部操纵

Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensing

In this paper, we present a novel method for mobile manipulators to perform
multiple contact-rich manipulation tasks. While learning-based methods have the
potential to generate actions in an end-to-end manner, they often suffer from
insufficient action accuracy and robustness against noise. On the other hand,
classical control-based methods can enhance system robustness, but at the cost
of extensive parameter tuning. To address these challenges, we present
MOMA-Force, a visual-force imitation method that seamlessly combines
representation learning for perception, imitation learning for complex motion
generation, and admittance whole-body control for system robustness and
controllability. MOMA-Force enables a mobile manipulator to learn multiple
complex contact-rich tasks with high success rates and small contact forces. In
a real household setting, our method outperforms baseline methods in terms of
task success rates. Moreover, our method achieves smaller contact forces and
smaller force variances compared to baseline methods without force imitation.
Overall, we offer a promising approach for efficient and robust mobile
manipulation in the real world. Videos and more details can be found on
https://visual-force-imitation.github.io

我们提出了一种新颖的方法，通过 MOMA-Force 实现了移动操纵器在多个接触丰富的操作任务中的高效和稳健的操纵。

MOMA-Force：实际移动操纵的视觉力模仿

MOMA-Force: Visual-Force Imitation for Real-World Mobile Manipulation

Contact-rich manipulation tasks in unstructured environments often require
both haptic and visual feedback. However, it is non-trivial to manually design
a robot controller that combines modalities with very different
characteristics. While deep reinforcement learning has shown success in
learning control policies for high-dimensional inputs, these algorithms are
generally intractable to deploy on real robots due to sample complexity. We use
self-supervision to learn a compact and multimodal representation of our
sensory inputs, which can then be used to improve the sample efficiency of our
policy learning. We evaluate our method on a peg insertion task, generalizing
over different geometry, configurations, and clearances, while being robust to
external perturbations. Results for simulated and real robot experiments are
presented.

本文使用自我监督的方式学习了一种紧凑的多模态表示方法，将触觉和视觉反馈结合起来，以改善高维输入控制策略的样本效率，该方法在模拟和实际机器人实验中均表现出鲁棒性和广泛泛化能力。