This paper proposes Panoptic Narrative Grounding, a spatially fine and
general formulation of the natural language visual grounding problem. We
establish an experimental framework for the study of this new task, including
new ground truth and metrics, and we propose a strong baseline method to serve
as stepping stone for future work. We exploit the intrinsic semantic richness
in an image by including panoptic categories, and we approach visual grounding
at a fine-grained level by using segmentations. In terms of ground truth, we
propose an algorithm to automatically transfer Localized Narratives annotations
to specific regions in the panoptic segmentations of the MS COCO dataset. To
guarantee the quality of our annotations, we take advantage of the semantic
structure contained in WordNet to exclusively incorporate noun phrases that are
grounded to a meaningfully related panoptic segmentation region. The proposed
baseline achieves a performance of 55.4 absolute Average Recall points. This
result is a suitable foundation to push the envelope further in the development
of methods for Panoptic Narrative Grounding.

本文提出了一种名为全景叙事基础的新型任务，包括新的实验框架、地面真值和指标，并提出了一个强大的基线方法。文章提出了一种算法，将分布式叙事注释传输到 MS COCO 数据集的普透明分割的特定区域。通过使用分割，文中还以精细的层次接近视觉基础问题，并通过包括全景类别来实现图像的内在语义丰富度。