We study the problem of weakly supervised grounded image captioning. That is,
given an image, the goal is to automatically generate a sentence describing the
context of the image with each noun word grounded to the corresponding region
in the image. This task is challenging due to the lack of explicit fine-grained
region word alignments as supervision. Previous weakly supervised methods
mainly explore various kinds of regularization schemes to improve attention
accuracy. However, their performances are still far from the fully supervised
ones. One main issue that has been ignored is that the attention for generating
visually groundable words may only focus on the most discriminate parts and can
not cover the whole object. To this end, we propose a simple yet effective
method to alleviate the issue, termed as partial grounding problem in our
paper. Specifically, we design a distributed attention mechanism to enforce the
network to aggregate information from multiple spatially different regions with
consistent semantics while generating the words. Therefore, the union of the
focused region proposals should form a visual region that encloses the object
of interest completely. Extensive experiments have demonstrated the superiority
of our proposed method compared with the state-of-the-arts.

该研究针对弱监督概念下的图片描述问题，提出了一种基于分布式注意力机制和局部连接信息的部分地基准技术，可以帮助生成最佳的图片描述。