Measuring visual similarity is critical for image understanding. But what
makes two images similar? Most existing work on visual similarity assumes that
images are similar because they contain the same object instance or category.
However, the reason why images are similar is much more complex. For example,
from the perspective of category, a black dog image is similar to a white dog
image. However, in terms of color, a black dog image is more similar to a black
horse image than the white dog image. This example serves to illustrate that
visual similarity is ambiguous but can be made precise when given an explicit
contextual perspective. Based on this observation, we propose the concept of
contextual visual similarity. To be concrete, we examine the concept of
contextual visual similarity in the application domain of image search. Instead
of providing only a single image for image similarity search (\eg, Google image
search), we require three images. Given a query image, a second positive image
and a third negative image, dissimilar to the first two images, we define a
contextualized similarity search criteria. In particular, we learn feature
weights over all the feature dimensions of each image such that the distance
between the query image and the positive image is small and their distances to
the negative image are large after reweighting their features. The learned
feature weights encode the contextualized visual similarity specified by the
user and can be used for attribute specific image search. We also show the
usefulness of our contextualized similarity weighting scheme for different
tasks, such as answering visual analogy questions and unsupervised attribute
discovery.

本文提出了概念 “上下文视觉相似度”，并探讨了在图像搜索应用领域的应用。使用三张图片，通过学习各个图像各个维度的特征权重以及重加权后的距离来确定查询图片与正向图片的相似度，以及他们与负向图片的相异度。