Large-scale generative models have demonstrated impressive capacity in
producing visually compelling images, with increasing applications in medical
imaging. However, they continue to grapple with the challenge of image
hallucination and the generation of anatomically inaccurate outputs. These
limitations are mainly due to the sole reliance on textual inputs and lack of
spatial control over the generated images, hindering the potential usefulness
of such models in real-life settings. We present XReal, a novel controllable
diffusion model for generating realistic chest X-ray images through precise
anatomy and pathology location control. Our lightweight method can seamlessly
integrate spatial control in a pre-trained text-to-image diffusion model
without fine-tuning, retaining its existing knowledge while enhancing its
generation capabilities. XReal outperforms state-of-the-art x-ray diffusion
models in quantitative and qualitative metrics while showing 13% and 10%
anatomy and pathology realism gain, respectively, based on the expert
radiologist evaluation. Our model holds promise for advancing generative models
in medical imaging, offering greater precision and adaptability while inviting
further exploration in this evolving field. A large synthetically generated
data with annotations and code is publicly available at
this https URL

通过准确的解剖学和病理定位控制，我们提出一种名为 XReal 的新型可控扩散模型，用于生成逼真的胸部 X 射线图像。我们的轻量级方法可以在预训练的文本到图像扩散模型中无需微调地无缝集成空间控制，保留其现有知识同时增强生成能力。XReal 在定量和定性指标上优于最先进的 X 射线扩散模型，根据专家放射科医师评估，解剖和病理逼真性分别提高了 13％和 10％。我们的模型有望在医学成像中推进生成模型，提供更高的准确性和适应性，并在这个不断发展的领域引发进一步的探索。大规模合成生成的带有注释和代码的数据公开可用于 this https URL。

XReal：基于可控扩散模型的逼真解剖与病理感知的 X 射线生成

XReal: Realistic Anatomy and Pathology-Aware X-ray Generation via  Controllable Diffusion Model

The field of deep generative modeling has grown rapidly and consistently over
the years. With the availability of massive amounts of training data coupled
with advances in scalable unsupervised learning paradigms, recent large-scale
generative models show tremendous promise in synthesizing high-resolution
images and text, as well as structured data such as videos and molecules.
However, we argue that current large-scale generative AI models do not
sufficiently address several fundamental issues that hinder their widespread
adoption across domains. In this work, we aim to identify key unresolved
challenges in modern generative AI paradigms that should be tackled to further
enhance their capabilities, versatility, and reliability. By identifying these
challenges, we aim to provide researchers with valuable insights for exploring
fruitful research directions, thereby fostering the development of more robust
and accessible generative AI solutions.

我们旨在识别现代生成 AI 范例中尚未解决的主要挑战，以进一步增强其能力、多样性和可靠性，并为研究人员提供有价值的见解，以探索更有成效的研究方向，从而促进更强大和可访问的生成 AI 解决方案的发展。

生成人工智能的挑战与机遇

On the Challenges and Opportunities in Generative AI

Large-scale generative models are shown to be useful for sampling meaningful
candidate solutions, yet they often overlook task constraints and user
preferences. Their full power is better harnessed when the models are coupled
with external verifiers and the final solutions are derived iteratively or
progressively according to the verification feedback. In the context of
embodied AI, verification often solely involves assessing whether goal
conditions specified in the instructions have been met. Nonetheless, for these
agents to be seamlessly integrated into daily life, it is crucial to account
for a broader range of constraints and preferences beyond bare task success
(e.g., a robot should grasp bread with care to avoid significant deformations).
However, given the unbounded scope of robot tasks, it is infeasible to
construct scripted verifiers akin to those used for explicit-knowledge tasks
like the game of Go and theorem proving. This begs the question: when no sound
verifier is available, can we use large vision and language models (VLMs),
which are approximately omniscient, as scalable Behavior Critics to catch
undesirable robot behaviors in videos? To answer this, we first construct a
benchmark that contains diverse cases of goal-reaching yet undesirable robot
policies. Then, we comprehensively evaluate VLM critics to gain a deeper
understanding of their strengths and failure modes. Based on the evaluation, we
provide guidelines on how to effectively utilize VLM critiques and showcase a
practical way to integrate the feedback into an iterative process of policy
refinement. The dataset and codebase are released at:
this https URL

在具体 AI 领域，利用大规模生成模型结合外部验证者，根据验证反馈逐步迭代推导最终解决方案，以验证是否达到说明中的目标条件，以便无缝整合到日常生活中，超越任务成功，和大范围的约束和个人偏好，为此构建一套测试基准，通过全面评估视觉与语言模型在识别视频中不良机器人行为方面的优点和失效模式，提供了有效利用模型评论的指导方针，并展示了将反馈融入政策改进的迭代过程的实用方法。