The hubness problem widely exists in high-dimensional embedding space and is
a fundamental source of error for cross-modal matching tasks. In this work, we
study the emergence of hubs in Visual Semantic Embeddings (VSE) with
application to text-image matching. We analyze the pros and cons of two widely
adopted optimization objectives for training VSE and propose a novel
hubness-aware loss function (HAL) that addresses previous methods' defects.
Unlike (Faghri et al.2018) which simply takes the hardest sample within a
mini-batch, HAL takes all samples into account, using both local and global
statistics to scale up the weights of "hubs". We experiment our method with
various configurations of model architectures and datasets. The method exhibits
exceptionally good robustness and brings consistent improvement on the task of
text-image matching across all settings. Specifically, under the same model
architectures as (Faghri et al. 2018) and (Lee at al. 2018), by switching only
the learning objective, we report a maximum R@1improvement of 7.4% on MS-COCO
and 8.3% on Flickr30k.

本文针对视觉与语义嵌入中的 hub 问题，探讨了两种优化目标以及提出的 hubness-aware loss function 的优点，并在模型架构和数据集方面进行了实验，结果表明该方法在 text-image matching 任务中具有优良的鲁棒性并且能够带来一致性的改进。

HAL: 通过缓解视觉语义中心改进文本图像匹配

HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs

This work aims to address the problem of image-based question-answering (QA)
with new models and datasets. In our work, we propose to use neural networks
and visual semantic embeddings, without intermediate stages such as object
detection and image segmentation, to predict answers to simple questions about
images. Our model performs 1.8 times better than the only published results on
an existing image QA dataset. We also present a question generation algorithm
that converts image descriptions, which are widely available, into QA form. We
used this algorithm to produce an order-of-magnitude larger dataset, with more
evenly distributed answers. A suite of baseline results on this new dataset are
also presented.

本研究旨在通过新的模型和数据集解决基于图像的问答问题，使用神经网络和视觉语义嵌入预测有关图像的简单问题的答案，提出了一种将图片描述转化为问答格式的问题生成算法，为新的数据集提供了基线结果。