The surge of Multimodal Large Language Models (MLLMs), given their prominent
emergent capabilities in instruction following and reasoning, has greatly
advanced the field of visual reasoning. However, constrained by their
non-lossless image tokenization, most MLLMs fall short of comprehensively
capturing details of text and objects, especially in high-resolution images. To
address this, we propose P2G, a novel framework for plug-and-play grounding of
reasoning in MLLMs. Specifically, P2G exploits the tool-usage potential of
MLLMs to employ expert agents to achieve on-the-fly grounding to critical
visual and textual objects of image, thus achieving deliberate reasoning via
multimodal prompting. We further create P2GB, a benchmark aimed at assessing
MLLMs' ability to understand inter-object relationships and text in challenging
high-resolution images. Comprehensive experiments on visual reasoning tasks
demonstrate the superiority of P2G. Noteworthy, P2G achieved comparable
performance with GPT-4V on P2GB, with a 7B backbone. Our work highlights the
potential of plug-and-play grounding of reasoning and opens up a promising
alternative beyond model scaling.

插拔式推理引擎 P2G 在多模态大型语言模型中的可视化推理任务表现优异，特别是在高分辨率图像中的文本和对象细节捕捉方面，与 GPT-4V 的性能相当，为模型扩展之外的有前景的替代方案。