Recent developments in diffusion models have advanced conditioned image
generation, yet they struggle with reconstructing out-of-distribution (OOD)
images, such as unseen tumors in medical images, causing ``image
hallucination'' and risking misdiagnosis. We hypothesize such hallucinations
result from local OOD regions in the conditional images. We verify that
partitioning the OOD region and conducting separate image generations
alleviates hallucinations in several applications. From this, we propose a
training-free diffusion framework that reduces hallucination with multiple
Local Diffusion processes. Our approach involves OOD estimation followed by two
modules: a ``branching'' module generates locally both within and outside OOD
regions, and a ``fusion'' module integrates these predictions into one. Our
evaluation shows our method mitigates hallucination over baseline models
quantitatively and qualitatively, reducing misdiagnosis by 40% and 25% in the
real-world medical and natural image datasets, respectively. It also
demonstrates compatibility with various pre-trained diffusion models.

通过局部超出分布区域划分和分离图像生成过程，本研究提出了一个无需训练的扩散模型，有效减少图像幻觉和误诊，减少真实世界医学和自然图像数据集中的误诊率分别达到 40% 和 25%，同时与各种预训练的扩散模型兼容。

用局部扩散解决图像翻译中的结构幻觉

Tackling Structural Hallucination in Image Translation with Local  Diffusion

Large-scale generative models have demonstrated impressive capacity in
producing visually compelling images, with increasing applications in medical
imaging. However, they continue to grapple with the challenge of image
hallucination and the generation of anatomically inaccurate outputs. These
limitations are mainly due to the sole reliance on textual inputs and lack of
spatial control over the generated images, hindering the potential usefulness
of such models in real-life settings. We present XReal, a novel controllable
diffusion model for generating realistic chest X-ray images through precise
anatomy and pathology location control. Our lightweight method can seamlessly
integrate spatial control in a pre-trained text-to-image diffusion model
without fine-tuning, retaining its existing knowledge while enhancing its
generation capabilities. XReal outperforms state-of-the-art x-ray diffusion
models in quantitative and qualitative metrics while showing 13% and 10%
anatomy and pathology realism gain, respectively, based on the expert
radiologist evaluation. Our model holds promise for advancing generative models
in medical imaging, offering greater precision and adaptability while inviting
further exploration in this evolving field. A large synthetically generated
data with annotations and code is publicly available at
this https URL

通过准确的解剖学和病理定位控制，我们提出一种名为 XReal 的新型可控扩散模型，用于生成逼真的胸部 X 射线图像。我们的轻量级方法可以在预训练的文本到图像扩散模型中无需微调地无缝集成空间控制，保留其现有知识同时增强生成能力。XReal 在定量和定性指标上优于最先进的 X 射线扩散模型，根据专家放射科医师评估，解剖和病理逼真性分别提高了 13％和 10％。我们的模型有望在医学成像中推进生成模型，提供更高的准确性和适应性，并在这个不断发展的领域引发进一步的探索。大规模合成生成的带有注释和代码的数据公开可用于 this https URL。