Humans seemingly incorporate potential touch signals in their perception. Our
goal is to equip robots with a similar capability, which we term Imagine2touch.
Imagine2touch aims to predict the expected touch signal based on a visual patch
representing the area to be touched. We use ReSkin, an inexpensive and compact
touch sensor to collect the required dataset through random touching of five
basic geometric shapes, and one tool. We train Imagine2touch on two out of
those shapes and validate it on the ood. tool. We demonstrate the efficacy of
Imagine2touch through its application to the downstream task of object
recognition. In this task, we evaluate Imagine2touch performance in two
experiments, together comprising 5 out of training distribution objects.
Imagine2touch achieves an object recognition accuracy of 58% after ten touches
per object, surpassing a proprioception baseline.

我们的目标是使机器人具备与人类类似的潜在触觉信号感知能力，通过使用一种名为 Imagine2touch 的方法，可以根据表示待触摸区域的视觉补丁来预测预期的触摸信号。我们使用了一种廉价而紧凑的触摸传感器 ReSkin 来收集所需的数据集，通过对五种基本几何形状和一种工具进行随机触摸。我们在其中两种形状上对 Imagine2touch 进行训练，并在 odd 工具上进行验证。我们通过将 Imagine2touch 应用于物体识别的下游任务来展示其有效性。在这个任务中，我们通过两个实验评估了 Imagine2touch 的性能，总共包括 5 个训练分布对象。在每个对象受到十次触摸后，Imagine2touch 的物体识别准确率达到了 58%，超过了一个本体感知基线。

Imagine2touch：使用低维信号的机器人操作预测性触觉感知

Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using  Efficient Low-Dimensional Signals

Humans seemingly incorporate potential touch signals in their perception. Our
goal is to equip robots with a similar capability, which we term \ourmodel.
\ourmodel aims to predict the expected touch signal based on a visual patch
representing the touched area. We frame this problem as the task of learning a
low-dimensional visual-tactile embedding, wherein we encode a depth patch from
which we decode the tactile signal. To accomplish this task, we employ ReSkin,
an inexpensive and replaceable magnetic-based tactile sensor. Using ReSkin, we
collect and train PseudoTouch on a dataset comprising aligned tactile and
visual data pairs obtained through random touching of eight basic geometric
shapes. We demonstrate the efficacy of PseudoTouch through its application to
two downstream tasks: object recognition and grasp stability prediction. In the
object recognition task, we evaluate the learned embedding's performance on a
set of five basic geometric shapes and five household objects. Using
PseudoTouch, we achieve an object recognition accuracy 84% after just ten
touches, surpassing a proprioception baseline. For the grasp stability task, we
use ACRONYM labels to train and evaluate a grasp success predictor using
PseudoTouch's predictions derived from virtual depth information. Our approach
yields an impressive 32% absolute improvement in accuracy compared to the
baseline relying on partial point cloud data. We make the data, code, and
trained models publicly available at this http URL

通过学习低维度的视觉 - 触觉嵌入，本研究利用触觉传感器和视觉数据实现了机器人的触觉信号预测，成功应用于物体识别和抓握稳定性预测任务，并在后两个任务上展现了优越性能。