When presented with questions involving visual thinking, humans naturally
switch reasoning modalities, often forming mental images or drawing visual
aids. Large language models have shown promising results in arithmetic and
symbolic reasoning by expressing intermediate reasoning in text as a chain of
thought, yet struggle to extend this capability to answer text queries that are
easily solved by visual reasoning, even with extensive multimodal pretraining.
We introduce a simple method, whiteboard-of-thought prompting, to unlock the
visual reasoning capabilities of multimodal large language models across
modalities. Whiteboard-of-thought prompting provides multimodal large language
models with a metaphorical `whiteboard' to draw out reasoning steps as images,
then returns these images back to the model for further processing. We find
this can be accomplished with no demonstrations or specialized modules, instead
leveraging models' existing ability to write code with libraries such as
Matplotlib and Turtle. This simple approach shows state-of-the-art results on
four difficult natural language tasks that involve visual and spatial
reasoning. We identify multiple settings where GPT-4o using chain-of-thought
fails dramatically, including more than one where it achieves $0\%$ accuracy,
while whiteboard-of-thought enables up to $92\%$ accuracy in these same
settings. We present a detailed exploration of where the technique succeeds as
well as its sources of error.

白板思维引导是一个简单的方法，可以解锁多模态大型语言模型在视觉推理方面的能力，通过提供 “白板” 以图像形式呈现推理步骤，然后将这些图像返回给模型进行进一步处理。该方法在涉及视觉和空间推理的四个自然语言任务方面展现了最先进的结果，同时解决了 GPT-4o 使用思维链的多个失败场景，并在这些相同场景中实现了 92% 的准确率。