Recognising the characteristics of objects while a robot handles them is
crucial for adjusting motions that ensure stable and efficient interactions
with containers. Ahead of realising stable and efficient robot motions for
handling/transferring the containers, this work aims to recognise the latent
unobservable object characteristics. While vision is commonly used for object
recognition by robots, it is ineffective for detecting hidden objects. However,
recognising objects indirectly using other sensors is a challenging task. To
address this challenge, we propose a cross-modal transfer learning approach
from vision to haptic-audio. We initially train the model with vision, directly
observing the target object. Subsequently, we transfer the latent space learned
from vision to a second module, trained only with haptic-audio and motor data.
This transfer learning framework facilitates the representation of object
characteristics using indirect sensor data, thereby improving recognition
accuracy. For evaluating the recognition accuracy of our proposed learning
framework we selected shape, position, and orientation as the object
characteristics. Finally, we demonstrate online recognition of both trained and
untrained objects using the humanoid robot Nextage Open.

通过视觉传感器和触觉 - 音频传感器的交叉模态迁移学习框架，提高物体识别准确性和机器人运动稳定性。