Large vision language models (LVLMs) integrate large language models (LLMs)
with pre-trained vision encoders, thereby activating the perception capability
of the model to understand image inputs for different queries and conduct
subsequent reasoning. Improving this capability requires high-quality
vision-language data, which is costly and labor-intensive to acquire.
Self-training approaches have been effective in single-modal settings to
alleviate the need for labeled data by leveraging model's own generation.
However, effective self-training remains a challenge regarding the unique
visual perception and reasoning capability of LVLMs. To address this, we
introduce Self-Training on Image Comprehension (STIC), which emphasizes a
self-training approach specifically for image comprehension. First, the model
self-constructs a preference dataset for image descriptions using unlabeled
images. Preferred responses are generated through a step-by-step prompt, while
dis-preferred responses are generated from either corrupted images or
misleading prompts. To further self-improve reasoning on the extracted visual
information, we let the model reuse a small portion of existing
instruction-tuning data and append its self-generated image descriptions to the
prompts. We validate the effectiveness of STIC across seven different
benchmarks, demonstrating substantial performance gains of 4.0% on average
while using 70% less supervised fine-tuning data than the current method.
Further studies investigate various components of STIC and highlight its
potential to leverage vast quantities of unlabeled images for self-training.
Code and data are made publicly available.

采用自我训练方法提高大型视觉语言模型在图像理解方面的能力，通过自动生成偏好图像描述来构建图像理解的偏好数据集，利用少量的现有数据进行自我改进，验证了其在七个不同基准测试中的有效性和潜力。