Image search engines rely on appropriately designed ranking features that
capture various aspects of the content semantics as well as the historic
popularity. In this work, we consider the role of colour in this relevance
matching process. Our work is motivated by the observation that a significant
fraction of user queries have an inherent colour associated with them. While
some queries contain explicit colour mentions (such as 'black car' and 'yellow
daisies'), other queries have implicit notions of colour (such as 'sky' and
'grass'). Furthermore, grounding queries in colour is not a mapping to a single
colour, but a distribution in colour space. For instance, a search for 'trees'
tends to have a bimodal distribution around the colours green and brown. We
leverage historical clickthrough data to produce a colour representation for
search queries and propose a recurrent neural network architecture to encode
unseen queries into colour space. We also show how this embedding can be learnt
alongside a cross-modal relevance ranker from impression logs where a subset of
the result images were clicked. We demonstrate that the use of a query-image
colour distance feature leads to an improvement in the ranker performance as
measured by users' preferences of clicked versus skipped images.

本文探讨了如何在图像搜索引擎中应用颜色作为匹配过程的一个特征，提出了一种基于循环神经网络及历史点击数据学习查询 - 图像颜色分布表示并结合交叉模式下的匹配模型，并在用户反馈中验证其有效性。