Visual program synthesis is a promising approach to exploit the reasoning
abilities of large language models for compositional computer vision tasks.
Previous work has used few-shot prompting with frozen LLMs to synthesize visual
programs. Training an LLM to write better visual programs is an attractive
prospect, but it is unclear how to accomplish this. No dataset of visual
programs for training exists, and acquisition of a visual program dataset
cannot be easily crowdsourced due to the need for expert annotators. To get
around the lack of direct supervision, we explore improving the program
synthesis abilities of an LLM using feedback from interactive experience. We
propose a method where we exploit existing annotations for a vision-language
task to improvise a coarse reward signal for that task, treat the LLM as a
policy, and apply reinforced self-training to improve the visual program
synthesis ability of the LLM for that task. We describe a series of experiments
on object detection, compositional visual question answering, and image-text
retrieval, and show that in each case, the self-trained LLM outperforms or
performs on par with few-shot frozen LLMs that are an order of magnitude
larger. Website: this https URL

利用交互式经验反馈改进大规模语言模型的视觉程序合成能力，通过利用现有的视觉语言任务注释为该任务创造一个粗略的奖励信号，将语言模型作为一种策略，并应用增强的自训练，显示出在对象检测、复合视觉问答和图像 - 文本检索方面，经过自训练的语言模型在每种情况下优于或与量级更大的少样本冻结的语言模型相媲美。