We present DualFocus, a novel framework for integrating macro and micro
perspectives within multi-modal large language models (MLLMs) to enhance
vision-language task performance. Current MLLMs typically singularly focus on
inputs at a predefined resolution, resulting in deficiencies in detailed
questions involving local regions. We introduced a DualFocus mechanism where
the model concentrates on the image from a macro perspective, responses to the
question, and identifies suitable sub-regions to zoom in for subsequent micro
perspective analysis. Via the integration of answers from both macro and micro
perspectives, the model is adept at addressing tasks that encompass global,
detailed, and combined considerations. To endows the DualFocus mechanism in
MLLMs, we curated a tailored dataset derived from the Visual Genome (VG) and
adapted it to align with the training regimen of DualFocus. Through comparative
studies across different model sizes and benchmarks, we demonstrate DualFocus's
superiority in balancing detailed examination with holistic insight,
significantly reducing hallucination instances in MLLMs and improving their
performance in various vision-language tasks.

通过在多模态大型语言模型中引入双重聚焦机制，该研究提出了一种新颖的框架，能够提升视觉 - 语言任务的性能，并在综合考虑全局、细节和综合因素的任务中展现出卓越优势。利用来自宏观和微观视角的图像信息和问题响应，该模型通过识别合适的子区域进行深入分析，从而实现对局部区域的详细问题的解答能力，从而减少了大型语言模型中的幻觉现象，并提高了各种视觉 - 语言任务的性能。