Current captioning approaches can describe images using black-box architectures whose behavior is hardly controllable and explainable from the exterior. As an image can be described in infinite ways depending on the goal and the context at hand, a higher degree of controllability is needed to apply captioning algorithms in complex scenarios. In this paper, we introduce a novel framework for image captioning which can generate diverse descriptions by allowing both grounding and controllability. Given a control signal in the form of a sequence or set of image regions, we generate the corresponding caption through a recurrent architecture which predicts textual chunks explicitly grounded on regions, following the constraints of the given control. Experiments are conducted on Flickr30k Entities and on COCO Entities, an extended version of COCO in which we add grounding annotations collected in a semi-automatic manner. Results demonstrate that our method achieves state of the art performances on controllable image captioning, in terms of caption quality and diversity. Code will be made publicly available.

本文介绍了一种新的图像字幕生成框架，该框架包括具有预测文本块和约束条件的递归架构，这些约束条件是通过控制信号（图像区域的序列或集合）确定的，并通过允许地面化和可控性来产生多样化的描述。实验表明，该方法在可控的图像字幕生成方面取得了最先进的性能。

Show, Control and Tell: 一种生成可控和有基础的字幕的框架