Image description datasets play a crucial role in the advancement of various
applications such as image understanding, text-to-image generation, and
text-image retrieval. Currently, image description datasets primarily originate
from two sources. One source is the scraping of image-text pairs from the web.
Despite their abundance, these descriptions are often of low quality and noisy.
Another is through human labeling. Datasets such as COCO are generally very
short and lack details. Although detailed image descriptions can be annotated
by humans, the high annotation cost limits the feasibility. These limitations
underscore the need for more efficient and scalable methods to generate
accurate and detailed image descriptions. In this paper, we propose an
innovative framework termed Image Textualization (IT), which automatically
produces high-quality image descriptions by leveraging existing multi-modal
large language models (MLLMs) and multiple vision expert models in a
collaborative manner, which maximally convert the visual information into text.
To address the current lack of benchmarks for detailed descriptions, we propose
several benchmarks for comprehensive evaluation, which verifies the quality of
image descriptions created by our framework. Furthermore, we show that
LLaVA-7B, benefiting from training on IT-curated descriptions, acquire improved
capability to generate richer image descriptions, substantially increasing the
length and detail of their output with less hallucination.

通过结合现有的多模态大语言模型（MLLMs）与多个视觉专家模型，我们提出了一种名为图像文本化（IT）的创新框架，以最大程度地将视觉信息转化为文本，自动生成高质量的图像描述。我们还提出了几个综合评估基准来验证我们框架所创建的图像描述的质量，显示经过 IT 精选描述训练的 LLaVA-7B 模型在生成更丰富的图像描述方面的能力得到了提高。

图像文本化：一种自动创建准确详细图像描述的框架

Image Textualization: An Automatic Framework for Creating Accurate and  Detailed Image Descriptions

Automatic image description systems are commonly trained and evaluated using
crowdsourced, human-generated image descriptions. The best-performing system is
then determined using some measure of similarity to the reference data (BLEU,
Meteor, CIDER, etc). Thus, both the quality of the systems as well as the
quality of the evaluation depends on the quality of the descriptions. As
Section 2 will show, the quality of current image description datasets is
insufficient. I argue that there is a need for more detailed guidelines that
take into account the needs of visually impaired users, but also the
feasibility of generating suitable descriptions. With high-quality data,
evaluation of image description systems could use reference descriptions, but
we should also look for alternatives.

通过产生更好的数据和寻找替代方法来改进图像描述系统的评估，因为当前的图像描述数据集质量不足。需要更详细的指导方针来考虑视觉障碍用户的需求和生成适当的描述。