Humans seemingly incorporate potential touch signals in their perception. Our
goal is to equip robots with a similar capability, which we term \ourmodel.
\ourmodel aims to predict the expected touch signal based on a visual patch
representing the touched area. We frame this problem as the task of learning a
low-dimensional visual-tactile embedding, wherein we encode a depth patch from
which we decode the tactile signal. To accomplish this task, we employ ReSkin,
an inexpensive and replaceable magnetic-based tactile sensor. Using ReSkin, we
collect and train PseudoTouch on a dataset comprising aligned tactile and
visual data pairs obtained through random touching of eight basic geometric
shapes. We demonstrate the efficacy of PseudoTouch through its application to
two downstream tasks: object recognition and grasp stability prediction. In the
object recognition task, we evaluate the learned embedding's performance on a
set of five basic geometric shapes and five household objects. Using
PseudoTouch, we achieve an object recognition accuracy 84% after just ten
touches, surpassing a proprioception baseline. For the grasp stability task, we
use ACRONYM labels to train and evaluate a grasp success predictor using
PseudoTouch's predictions derived from virtual depth information. Our approach
yields an impressive 32% absolute improvement in accuracy compared to the
baseline relying on partial point cloud data. We make the data, code, and
trained models publicly available at this http URL

通过学习低维度的视觉 - 触觉嵌入，本研究利用触觉传感器和视觉数据实现了机器人的触觉信号预测，成功应用于物体识别和抓握稳定性预测任务，并在后两个任务上展现了优越性能。