This paper explores the image-sharing capability of Large Language Models
(LLMs), such as InstructGPT, ChatGPT, and GPT-4, in a zero-shot setting,
without the help of visual foundation models. Inspired by the two-stage process
of image-sharing in human dialogues, we propose a two-stage framework that
allows LLMs to predict potential image-sharing turns and generate related image
descriptions using our effective restriction-based prompt template. With
extensive experiments, we unlock the \textit{image-sharing} capability of LLMs
in zero-shot prompting, with GPT-4 achieving the best performance.
Additionally, we uncover the emergent \textit{image-sharing} ability in
zero-shot prompting, demonstrating the effectiveness of restriction-based
prompts in both stages of our framework. Based on this framework, we augment
the PhotoChat dataset with images generated by Stable Diffusion at predicted
turns, namely PhotoChat++. To our knowledge, this is the first study to assess
the image-sharing ability of LLMs in a zero-shot setting without visual
foundation models. The source code and the dataset will be released after
publication.

该研究探讨了大型语言模型（LLMs），如 InstructGPT，ChatGPT 和 GPT-4，在零样本设置中的图像共享能力，提出了一个两阶段框架，使 LLMs 能够预测潜在的图像共享转向并生成相关的图像描述，通过广泛的实验证明了 GPT-4 在零样本提示下实现了最佳性能，此外，我们发现了零样本提示中的紧密共享能力，证明了我们框架的两个阶段中基于限制的提示的有效性。基于该框架，我们利用 Stable Diffusion 在预测的转向处生成图像，即 PhotoChat ++，据我们所知，这是第一项在没有视觉基础模型的零样本设置中评估 LLMs 图像共享能力的研究。发表后将发布源代码和数据集。