In robotics and artificial intelligence, the integration of tactile
processing is becoming increasingly pivotal, especially in learning to execute
intricate tasks like alignment and insertion. However, existing works focusing
on tactile methods for insertion tasks predominantly rely on robot
teleoperation data and reinforcement learning, which do not utilize the rich
insights provided by human's control strategy guided by tactile feedback. For
utilizing human sensations, methodologies related to learning from humans
predominantly leverage visual feedback, often overlooking the invaluable
tactile feedback that humans inherently employ to finish complex manipulations.
Addressing this gap, we introduce "MimicTouch", a novel framework that mimics
human's tactile-guided control strategy. In this framework, we initially
collect multi-modal tactile datasets from human demonstrators, incorporating
human tactile-guided control strategies for task completion. The subsequent
step involves instructing robots through imitation learning using multi-modal
sensor data and retargeted human motions. To further mitigate the embodiment
gap between humans and robots, we employ online residual reinforcement learning
on the physical robot. Through comprehensive experiments, we validate the
safety of MimicTouch in transferring a latent policy learned through imitation
learning from human to robot. This ongoing work will pave the way for a broader
spectrum of tactile-guided robotic applications.

机器人和人工智能领域中，触觉处理的整合越来越关键，特别是在学习如对齿轮和插入等复杂任务时。然而，现有的关于插入任务的触觉方法的研究主要依赖机器人远程操作数据和强化学习，并没有充分利用人类在触觉反馈指导下的控制策略所提供的丰富见解。为了利用人类的感知，与学习源自人类的方法相比，方法学上主要利用视觉反馈，常常忽视人类在完成复杂操纵时固有地使用的宝贵触觉反馈。为填补这一差距，我们引入了一种名为 “MimicTouch” 的新框架，该框架模仿人类的触觉引导控制策略。在这个框架中，我们首先从人类示范者那里收集多模态触觉数据集，将人类的触觉引导控制策略融入到任务完成中。其后的步骤涉及使用多模态传感器数据和重新定位的人类动作通过模仿学习指导机器人。为了进一步缩小人类与机器人之间的差距，我们在物理机器人上采用在线剩余强化学习。通过全面的实验证明，在从人类到机器人的过程中，MimicTouch 的潜在策略通过模仿学习的方式转移是安全和可行的。这项正在进行的工作将为更广泛的触觉引导机器人应用铺平道路。