We explore multi-step reasoning in vision-language models (VLMs). The problem
is challenging, as reasoning data consisting of multiple steps of visual and
language processing are barely available. To overcome the challenge, we first
introduce a least-to-most visual reasoning paradigm, which interleaves steps of
decomposing a question into sub-questions and invoking external tools for
resolving sub-questions. Based on the paradigm, we further propose a novel data
synthesis approach that can automatically create questions and multi-step
reasoning paths for an image in a bottom-up manner. Our approach divides the
complex synthesis task into a few simple sub-tasks, and (almost entirely)
relies on open-sourced models to accomplish the sub-tasks. Therefore, the
entire synthesis process is reproducible and cost-efficient, and the
synthesized data is quality guaranteed. With the approach, we construct $50$k
visual reasoning examples. Then, we develop a visual reasoner through
supervised fine-tuning, which is capable of generally enhancing the reasoning
abilities of a wide range of existing VLMs in a plug-and-play fashion.
Extensive experiments indicate that the visual reasoner can consistently and
significantly improve four VLMs on four VQA benchmarks. Our code and dataset
are available at this https URL

我们探索了视觉 - 语言模型中的多步推理问题，并提出了一种新的数据合成方法，通过使用先生成较简单的子任务，并依赖开源模型来完成这些子任务，从而构建了 50000 个视觉推理示例，并通过监督微调开发了一种视觉推理器，可以显著改善四个视觉问答基准上的四个视觉 - 语言模型的推理能力。