Optical tactile sensors have emerged as an effective means to acquire dense
contact information during robotic manipulation. A recently-introduced
`see-through-your-skin' (STS) variant of this type of sensor has both visual
and tactile modes, enabled by leveraging a semi-transparent surface and
controllable lighting. In this work, we investigate the benefits of pairing
visuotactile sensing with imitation learning for contact-rich manipulation
tasks. First, we use tactile force measurements and a novel algorithm during
kinesthetic teaching to yield a force profile that better matches that of the
human demonstrator. Second, we add visual/tactile STS mode switching as a
control policy output, simplifying the application of the sensor. Finally, we
study multiple observation configurations to compare and contrast the value of
visual/tactile data (both with and without mode switching) with visual data
from a wrist-mounted eye-in-hand camera. We perform an extensive series of
experiments on a real robotic manipulator with door-opening and closing tasks,
including over 3,000 real test episodes. Our results highlight the importance
of tactile sensing for imitation learning, both for data collection to allow
force matching, and for policy execution to allow accurate task feedback.

研究了视触觉感知与模仿学习相结合在接触丰富的操作任务中的作用，利用光学触觉传感器和创新算法收集触觉力量数据，并通过可视 / 触觉模式切换简化传感器的应用，实验结果强调触觉感知在模仿学习中的重要性。

将其推至示范极限：多模态视触力仿真学习与力匹配

Push it to the Demonstrated Limit: Multimodal Visuotactile Imitation  Learning with Force Matching

We introduce a Learning from Demonstration (LfD) approach for contact-rich
manipulation tasks with articulated mechanisms. The extracted policy from a
single human demonstration generalizes to different mechanisms of the same type
and is robust against environmental variations. The key to achieving such
generalization and robustness from a single human demonstration is to
autonomously augment the initial demonstration to gather additional information
through purposefully interacting with the environment. Our real-world
experiments on complex mechanisms with multi-DOF demonstrate that our approach
can reliably accomplish the task in a changing environment. Videos are
available at the: this https URL

文章介绍了一种学习自演示 (LfD) 方法，用于应对带有关节机制的接触丰富的操作任务。从单个人类演示中提取的策略可以推广到相同类型的不同机制，并且对环境变化具有鲁棒性，其关键是通过与环境有目的地交互来自主增强初始演示以收集附加信息。针对多 DOF 的复杂机制的实际实验表明，我们的方法可以可靠地完成任务。

基于环境限制的示教学习增强

Augmentation for Learning From Demonstration with Environmental Constraints

Reinforcement Learning (RL) of contact-rich manipulation tasks has yielded
impressive results in recent years. While many studies in RL focus on varying
the observation space or reward model, few efforts focused on the choice of
action space (e.g. joint or end-effector space, position, velocity, etc.).
However, studies in robot motion control indicate that choosing an action space
that conforms to the characteristics of the task can simplify exploration and
improve robustness to disturbances. This paper studies the effect of different
action spaces in deep RL and advocates for Variable Impedance Control in
End-effector Space (VICES) as an advantageous action space for constrained and
contact-rich tasks. We evaluate multiple action spaces on three prototypical
manipulation tasks: Path Following (task with no contact), Door Opening (task
with kinematic constraints), and Surface Wiping (task with continuous contact).
We show that VICES improves sample efficiency, maintains low energy
consumption, and ensures safety across all three experimental setups. Further,
RL policies learned with VICES can transfer across different robot models in
simulation, and from simulation to real for the same robot. Further information
is available at this https URL

本文研究了深度强化学习中不同行为空间的影响，提出了在约束和接触丰富任务中具有优势的终端执行器空间下的可变阻抗控制（VICES）。通过在三个典型的操作任务中评估多个动作空间，表明 VICES 提高了样本效率，在所有三个实验设置中保持低能量消耗，并确保安全。