Attention mechanisms have attracted considerable interest in image captioning
because of its powerful performance. Existing attention-based models use
feedback information from the caption generator as guidance to determine which
of the image features should be attended to. A common defect of these attention
generation methods is that they lack a higher-level guiding information from
the image itself, which sets a limit on selecting the most informative image
features. Therefore, in this paper, we propose a novel attention mechanism,
called topic-guided attention, which integrates image topics in the attention
model as a guiding information to help select the most important image
features. Moreover, we extract image features and image topics with separate
networks, which can be fine-tuned jointly in an end-to-end manner during
training. The experimental results on the benchmark Microsoft COCO dataset show
that our method yields state-of-art performance on various quantitative
metrics.

本文提出了一种新的注意力机制，称为主题引导注意力，在图像字幕生成中应用，通过将图像主题整合到注意力模型中作为指导信息，协助选择最重要的图像特征。在微软 COCO 数据集上的实验结果表明，该方法在各种定量指标上具有最先进的性能。

基于主题引导的图像描述注意力模型

Topic-Guided Attention for Image Captioning

We propose a novel weakly-supervised semantic segmentation algorithm based on
Deep Convolutional Neural Network (DCNN). Contrary to existing
weakly-supervised approaches, our algorithm exploits auxiliary segmentation
annotations available for different categories to guide segmentations on images
with only image-level class labels. To make the segmentation knowledge
transferrable across categories, we design a decoupled encoder-decoder
architecture with attention model. In this architecture, the model generates
spatial highlights of each category presented in an image using an attention
model, and subsequently generates foreground segmentation for each highlighted
region using decoder. Combining attention model, we show that the decoder
trained with segmentation annotations in different categories can boost the
performance of weakly-supervised semantic segmentation. The proposed algorithm
demonstrates substantially improved performance compared to the
state-of-the-art weakly-supervised techniques in challenging PASCAL VOC 2012
dataset when our model is trained with the annotations in 60 exclusive
categories in Microsoft COCO dataset.

该论文介绍了一种新颖的弱监督语义分割算法，该算法利用基于深度卷积神经网络的辅助分割注释，通过设计解耦编码器 - 解码器体系结构来生成空间高光区域，并使用注意力模型和分割注解在不同类型中加速弱监督下的语义分割。 训练该模型可以在 Microsoft COCO 数据集中的 60 个独家类别的注释下，显着提高在具有挑战性的 PASCAL VOC 2012 数据集中的表现。