Text-based image captioning (TextCap) which aims to read and reason images
with texts is crucial for a machine to understand a detailed and complex scene
environment, considering that texts are omnipresent in daily life. This task,
however, is very challenging because an image often contains complex texts and
visual information that is hard to be described comprehensively. Existing
methods attempt to extend the traditional image captioning methods to solve
this task, which focus on describing the overall scene of images by one global
caption. This is infeasible because the complex text and visual information
cannot be described well within one caption. To resolve this difficulty, we
seek to generate multiple captions that accurately describe different parts of
an image in detail. To achieve this purpose, there are three key challenges: 1)
it is hard to decide which parts of the texts of images to copy or paraphrase;
2) it is non-trivial to capture the complex relationship between diverse texts
in an image; 3) how to generate multiple captions with diverse content is still
an open problem. To conquer these, we propose a novel Anchor-Captioner method.
Specifically, we first find the important tokens which are supposed to be paid
more attention to and consider them as anchors. Then, for each chosen anchor,
we group its relevant texts to construct the corresponding anchor-centred graph
(ACG). Last, based on different ACGs, we conduct multi-view caption generation
to improve the content diversity of generated captions. Experimental results
show that our method not only achieves SOTA performance but also generates
diverse captions to describe images.

本文提出了一种基于锚文本和锚中心图的多视角多解释图像说明生成方法，以提高生成说明的多样性和准确性。