The well-known word analogy experiments show that the recent word vectors
capture fine-grained linguistic regularities in words by linear vector offsets,
but it is unclear how well the simple vector offsets can encode visual
regularities over words. We study a particular image-word relevance relation in
this paper. Our results show that the word vectors of relevant tags for a given
image rank ahead of the irrelevant tags, along a principal direction in the
word vector space. Inspired by this observation, we propose to solve image
tagging by estimating the principal direction for an image. Particularly, we
exploit linear mappings and nonlinear deep neural networks to approximate the
principal direction from an input image. We arrive at a quite versatile tagging
model. It runs fast given a test image, in constant time w.r.t.\ the training
set size. It not only gives superior performance for the conventional tagging
task on the NUS-WIDE dataset, but also outperforms competitive baselines on
annotating images with previously unseen tags

本篇论文研究了图像与单词之间的关系，发现相关的单词向量在特定方向上超过无关的单词向量，依此提出了一种基于线性映射和深层神经网络的快速图像标注模型，其具有良好的性能和鲁棒性，并能够处理此前未见过的标签。