Existing vision-language models (VLMs) often suffer from visual hallucination, where the generated responses contain inaccuracies that are not grounded in the visual input. Efforts to address this issue without model finetuning primarily mitigate hallucination by reducing biases contrastively or amplifying the weights of visual embedding during decoding. However, these approaches improve visual perception at the cost of impairing the language reasoning capability. In this work, we propose the Perception Magnifier (PM), a novel visual decoding method that iteratively isolates relevant visual tokens based on attention and magnifies the corresponding regions, spurring the model to concentrate on fine-grained visual details during decoding. Specifically, by magnifying critical regions while preserving the structural and contextual information at each decoding step, PM allows the VLM to enhance its scrutiny of the visual input, hence producing more accurate and faithful responses. Extensive experimental results demonstrate that PM not only achieves superior hallucination mitigation but also enhances language generation while preserving strong reasoning capabilities. Code is available at https://github.com/ShunqiM/PM .

本研究旨在解决现有视觉语言模型在解码过程中常出现的视觉幻觉问题。提出了一种新颖的视觉解码方法——感知放大器（PM），该方法通过迭代选择相关的视觉标记并放大相应区域，显著提高了解码过程中的视觉细节关注度。实验结果表明，PM不仅有效减少了幻觉现象，还增强了语言生成能力，同时保持了强大的推理能力。

透过放大镜：无幻觉的视觉语言模型解码的自适应感知放大