From paired image-text training to text-only training for image captioning,
the pursuit of relaxing the requirements for high-cost and large-scale
annotation of good quality data remains consistent. In this paper, we propose
Text-only Synthesis for Image Captioning (ToCa), which further advances this
relaxation with fewer human labor and less computing time. Specifically, we
deconstruct caption text into structures and lexical words, which serve as the
fundamental components of the caption. By combining different structures and
lexical words as inputs to the large language model, massive captions that
contain various patterns of lexical words are generated. This method not only
approaches the target domain but also surpasses it by generating new captions,
thereby enhancing the zero-shot generalization ability of the model.
Considering the different levels of data access in the real world, we define
three synthesis scenarios: cross-domain synthesis, in-domain synthesis, and
data-efficient synthesis. Experiments in these scenarios demonstrate the
generalizability, transferability and practicability of ToCa with a nearly 5
CIDEr improvement for zero-shot cross-domain captioning and a maximum increase
of over 20 CIDEr for data-efficient captioning.

提出了 ToCa 方法，通过将标题文本分解为结构和词汇单词，利用大型语言模型生成包含各种词汇模式的大量标题，从而实现对图像标题生成的非人力和计算成本的进一步放宽。