Our goal in this work is to train an image captioning model that generates
more dense and informative captions. We introduce "relational captioning," a
novel image captioning task which aims to generate multiple captions with
respect to relational information between objects in an image. Relational
captioning is a framework that is advantageous in both diversity and amount of
information, leading to image understanding based on relationships. Part-of
speech (POS, i.e. subject-object-predicate categories) tags can be assigned to
every English word. We leverage the POS as a prior to guide the correct
sequence of words in a caption. To this end, we propose a multi-task
triple-stream network (MTTSNet) which consists of three recurrent units for the
respective POS and jointly performs POS prediction and captioning. We
demonstrate more diverse and richer representations generated by the proposed
model against several baselines and competing methods.

通过引入 “关系字幕” 任务，使用多任务三流网络（MTTSNet）实现 POS 标签指导图像说明的正确顺序，从而提高图像理解能力，相应地，生成更多样化、更丰富的信息表达。