Recent advances in the field of intelligent robotic manipulation pursue
providing robotic hands with touch sensitivity. Haptic perception encompasses
the sensing modalities encountered in the sense of touch (e.g., tactile and
kinesthetic sensations). This letter focuses on multimodal object recognition
and proposes analytical and data-driven methodologies to fuse tactile- and
kinesthetic-based classification results. The procedure is as follows: a
three-finger actuated gripper with an integrated high-resolution tactile sensor
performs squeeze-and-release Exploratory Procedures (EPs). The tactile images
and kinesthetic information acquired using angular sensors on the finger joints
constitute the time-series datasets of interest. Each temporal dataset is fed
to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify
in-hand objects. The LSTMs provide an estimation of the posterior probability
of each object given the corresponding measurements, which after fusion allows
to estimate the object through Bayesian and Neural inference approaches. An
experiment with 36-classes is carried out to evaluate and compare the
performance of the fused, tactile, and kinesthetic perception systems.The
results show that the Bayesian-based classifiers improves capabilities for
object recognition and outperforms the Neural-based approach.

本研究提出了一种基于多模态对象识别的分析和数据驱动的方法，利用具有高分辨率触觉传感器的三指驱动夹爪进行压缩和释放探索过程（EPs），采集手指关节上的角度传感器获取触觉图像和动觉信息，然后，利用 LSTM 神经网络进行分类，最后通过贝叶斯和神经推理方法融合，对 36 种物体进行测试，结果表明贝叶斯分类器提高了物体识别能力并优于神经网络分类器。