Human infants learn the names of objects and develop their own conceptual
systems without explicit supervision. In this study, we propose methods for
learning aligned vision-language conceptual systems inspired by infants' word
learning mechanisms. The proposed model learns the associations of visual
objects and words online and gradually constructs cross-modal relational graph
networks. Additionally, we also propose an aligned cross-modal representation
learning method that learns semantic representations of visual objects and
words in a self-supervised manner based on the cross-modal relational graph
networks. It allows entities of different modalities with conceptually the same
meaning to have similar semantic representation vectors. We quantitatively and
qualitatively evaluate our method, including object-to-word mapping and
zero-shot learning tasks, showing that the proposed model significantly
outperforms the baselines and that each conceptual system is topologically
aligned.

本文提出了学习与婴儿词汇学习机制相似的视觉 - 语言概念系统的方法，包括了在线学习视觉对象和词汇之间的关联以及自监督学习的语义表示等。作者在目标词与词语匹配以及零样本学习任务中进行了定量和定性检验，结果表明该模型在性能方面优于其他基础模型并实现了概念系统的拓扑对齐。