The sense of touch is an essential ability for skillfully performing a
variety of tasks, providing the capacity to search and manipulate objects
without relying on visual information. Extensive research has been conducted
over time to apply these human tactile abilities to robots. In this paper, we
introduce a multi-finger robot system designed to search for and manipulate
objects using the sense of touch without relying on visual information.
Randomly located target objects are searched using tactile sensors, and the
objects are manipulated for tasks that mimic daily-life. The objective of the
study is to endow robots with human-like tactile capabilities. To achieve this,
binary tactile sensors are implemented on one side of the robot hand to
minimize the Sim2Real gap. Training the policy through reinforcement learning
in simulation and transferring the trained policy to the real environment, we
demonstrate that object search and manipulation using tactile sensors is
possible even in an environment without vision information. In addition, an
ablation study was conducted to analyze the effect of tactile information on
manipulative tasks. Our project page is available at
this https URL

通过使用触觉传感器而无需依赖视觉信息，我们介绍了一个多指机器人系统，用于搜索和操纵随机位置的目标物体，并在模拟环境中训练和转移政策，证明了使用触觉传感器进行物体搜索和操纵任务的可行性，以实现赋予机器人类似人类触觉能力的目标。

DexTouch：学习使用触觉灵巧性寻找和操纵物体

DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexterity

The sense of touch is essential for a variety of daily tasks. New advances in
event-based tactile sensors and Spiking Neural Networks (SNNs) spur the
research in event-driven tactile learning. However, SNN-enabled event-driven
tactile learning is still in its infancy due to the limited representative
abilities of existing spiking neurons and high spatio-temporal complexity in
the data. In this paper, to improve the representative capabilities of existing
spiking neurons, we propose a novel neuron model called "location spiking
neuron", which enables us to extract features of event-based data in a novel
way. Moreover, based on the classical Time Spike Response Model (TSRM), we
develop a specific location spiking neuron model - Location Spike Response
Model (LSRM) that serves as a new building block of SNNs. Furthermore, we
propose a hybrid model which combines an SNN with TSRM neurons and an SNN with
LSRM neurons to capture the complex spatio-temporal dependencies in the data.
Extensive experiments demonstrate the significant improvements of our models
over other works on event-driven tactile learning and show the superior energy
efficiency of our models and location spiking neurons, which may unlock their
potential on neuromorphic hardware.

本研究提出一种新的神经元模型 “位置尖峰神经元”，用以提取事件数据特征，同时将其与时间尖峰响应模型相结合，以构建事件驱动触觉学习的混合模型。实验表明，该模型在提高触觉学习效果和节能方面有着显著的优越性。

带有位置尖峰神经元的事件驱动触觉学习

Event-Driven Tactile Learning with Location Spiking Neurons

The sense of touch, being the earliest sensory system to develop in a human
body [1], plays a critical part of our daily interaction with the environment.
In order to successfully complete a task, many manipulation interactions
require incorporating haptic feedback. However, manually designing a feedback
mechanism can be extremely challenging. In this work, we consider manipulation
tasks that need to incorporate tactile sensor feedback in order to modify a
provided nominal plan. To incorporate partial observation, we present a new
framework that models the task as a partially observable Markov decision
process (POMDP) and learns an appropriate representation of haptic feedback
which can serve as the state for a POMDP model. The model, that is parametrized
by deep recurrent neural networks, utilizes variational Bayes methods to
optimize the approximate posterior. Finally, we build on deep Q-learning to be
able to select the optimal action in each state without access to a simulator.
We test our model on a PR2 robot for multiple tasks of turning a knob until it
clicks.

本文提出了一种将触觉感知反馈模型建模成部分可观察 Markov 决策过程并利用深度递归神经网络优化来进行任务操作的方法，其可以用于完成机械臂旋转旋钮等多种指定任务。