Modern image captioning system relies heavily on extracting knowledge from
images to capture the concept of a static story. In this paper, we propose a
textual visual context dataset for captioning, in which the publicly available
dataset COCO Captions (Lin et al., 2014) has been extended with information
about the scene (such as objects in the image). Since this information has a
textual form, it can be used to leverage any NLP task, such as text similarity
or semantic relation methods, into captioning systems, either as an end-to-end
training strategy or a post-processing based approach.

本论文介绍了一种文本视觉背景数据集，用于改进现代图像描述系统，通过融合与场景相关的文本信息，提高图像描述的准确性及语义关联性。

图像字幕视觉语义关联数据集

Visual Semantic Relatedness Dataset for Image Captioning

The de-facto approach to many vision tasks is to start from pretrained visual
representations, typically learned via supervised training on ImageNet. Recent
methods have explored unsupervised pretraining to scale to vast quantities of
unlabeled images. In contrast, we aim to learn high-quality visual
representations from fewer images. To this end, we revisit supervised
pretraining, and seek data-efficient alternatives to classification-based
pretraining. We propose VirTex -- a pretraining approach using semantically
dense captions to learn visual representations. We train convolutional networks
from scratch on COCO Captions, and transfer them to downstream recognition
tasks including image classification, object detection, and instance
segmentation. On all tasks, VirTex yields features that match or exceed those
learned on ImageNet -- supervised or unsupervised -- despite using up to ten
times fewer images.

使用 COCO Captions 监督预训练的 VirTex 方法可以在使用更少的图像的情况下，产生与 ImageNet 监督或非监督学习得到的特征所匹配或超越的视觉表征。