Text-only Image Captioning (TIC) is an approach that aims to construct a
model solely based on text that can accurately describe images. Recently,
diffusion models have demonstrated remarkable capabilities in generating
high-quality images that are semantically coherent with given texts. This
presents an opportunity to generate synthetic training images for TIC. However,
we have identified a challenge that the images generated from simple
descriptions typically exhibit a single perspective with one or limited
contexts, which is not aligned with the complexity of real-world scenes in the
image domain. In this paper, we propose a novel framework that addresses this
issue by introducing multi-context data generation. Starting with an initial
text corpus, our framework employs a large language model to select multiple
sentences that describe the same scene from various perspectives. These
sentences are then summarized into a single sentence with multiple contexts. We
generate simple images using the straightforward sentences and complex images
using the summarized sentences through diffusion models. Finally, we train the
model exclusively using the synthetic image-text pairs obtained from this
process. Experimental results demonstrate that our proposed framework
effectively tackles the central challenge we have identified, achieving the
state-of-the-art performance on popular datasets such as MSCOCO, Flickr30k, and
SS1M.

本文提出了一种新的多情景数据生成框架用于提高文本图像注释的训练数据，该框架使用扩散模型生成复杂和简单图像，并在 MSCOCO、Flickr30k 和 SS1M 等数据集上实现了最先进的表现。