Phrase grounding, the problem of associating image regions to caption words,
is a crucial component of vision-language tasks. We show that phrase grounding
can be learned by optimizing word-region attention to maximize a lower bound on
mutual information between images and caption words. Given pairs of images and
captions, we maximize compatibility of the attention-weighted regions and the
words in the corresponding caption, compared to non-corresponding pairs of
images and captions. A key idea is to construct effective negative captions for
learning through language model guided word substitutions. Training with our
negatives yields a $\sim10\%$ absolute gain in accuracy over randomly-sampled
negatives from the training data. Our weakly supervised phrase grounding model
trained on COCO-Captions shows a healthy gain of $5.7\%$ to achieve $76.7\%$
accuracy on Flickr30K Entities benchmark.

本文介绍了一种通过优化单词 - 区域关注力最大化互信息的方法来学习短语定位，该方法通过利用经过语言模型引导的单词替换来构建有效的负面标题进行训练，进而实现对 CO-Captions 数据集的短语 grounding，使精度提高了 5.7%。