Text-visual (or called semantic-visual) embedding is a central problem in
vision-language research. It typically involves mapping of an image and a text
description to a common feature space through a CNN image encoder and a RNN
language encoder. In this paper, we propose a new method for learning
text-visual embedding using both image titles and click-through data from an
image search engine. We also propose a new triplet loss function by modeling
positive awareness of the embedding, and introduce a novel mini-batch-based
hard negative sampling approach for better data efficiency in the learning
process. Experimental results show that our proposed method outperforms
existing methods, and is also effective for real-world text-to-visual
retrieval.

论文提出一种新的方法，使用图像标题和来自图像搜索引擎的点击数据来学习文本 - 视觉嵌入，并通过建模嵌入的积极感知提出新的三元损失函数，以及引入一种新的基于小批次的难例负采样方法来提高学习过程的数据效率，实验结果表明，该方法的表现优于现有方法，并且对于现实世界的文本到视觉检索也十分有效。