This paper introduces a novel approach to learn visually grounded meaning
representations of words as low-dimensional node embeddings on an underlying
graph hierarchy. The lower level of the hierarchy models modality-specific word
representations through dedicated but communicating graphs, while the higher
level puts these representations together on a single graph to learn a
representation jointly from both modalities. The topology of each graph models
similarity relations among words, and is estimated jointly with the graph
embedding. The assumption underlying this model is that words sharing similar
meaning correspond to communities in an underlying similarity graph in a
low-dimensional space. We named this model Hierarchical Multi-Modal Similarity
Graph Embedding (HM-SGE). Experimental results validate the ability of HM-SGE
to simulate human similarity judgements and concept categorization,
outperforming the state of the art.

本文提出了一种新的方法来学习基于视觉的单词含义表示，即在底层模态特定的词语表示上建立相互通信的图形，通过模拟人类相似性判断和概念分类来验证其能力，名为分层多模态相似性图嵌入（HM-SGE）。