An industrial connector insertion task requires submillimeter positioning and
grasp pose compensation for a plug. Thus, highly accurate estimation of the
relative pose between a plug and socket is fundamental for achieving the task.
World models are promising technologies for visuomotor control because they
obtain appropriate state representation to jointly optimize feature extraction
and latent dynamics model. Recent studies show that the NewtonianVAE, a type of
the world model, acquires latent space equivalent to mapping from images to
physical coordinates. Proportional control can be achieved in the latent space
of NewtonianVAE. However, applying NewtonianVAE to high-accuracy industrial
tasks in physical environments is an open problem. Moreover, the existing
framework does not consider the grasp pose compensation in the obtained latent
space. In this work, we proposed tactile-sensitive NewtonianVAE and applied it
to a USB connector insertion with grasp pose variation in the physical
environments. We adopted a GelSight-type tactile sensor and estimated the
insertion position compensated by the grasp pose of the plug. Our method trains
the latent space in an end-to-end manner, and no additional engineering and
annotation are required. Simple proportional control is available in the
obtained latent space. Moreover, we showed that the original NewtonianVAE fails
in some situations, and demonstrated that domain knowledge induction improves
model accuracy. This domain knowledge can be easily obtained using robot
specification and grasp pose error measurement. We demonstrated that our
proposed method achieved a 100\% success rate and 0.3 mm positioning accuracy
in the USB connector insertion task in the physical environment. It
outperformed SOTA CNN-based two-stage goal pose regression with grasp pose
compensation using coordinate transformation.

本文提出了一种基于触觉传感器的 NewtonianVAE 模型，结合机械臂插入任务中的握姿补偿，针对物理环境中插入 USB 插头的精确姿态定位问题进行了实验研究。实验结果表明，该模型在图像到物理坐标的映射等方面具有相对更高的准确性，相比基于卷积神经网络的传统两阶段目标姿态回归方法表现更为优异。