To achieve high-quality results, diffusion models must be trained on large
datasets. This can be notably prohibitive for models in specialized domains,
such as computational pathology. Conditioning on labeled data is known to help
in data-efficient model training. Therefore, histopathology reports, which are
rich in valuable clinical information, are an ideal choice as guidance for a
histopathology generative model. In this paper, we introduce PathLDM, the first
text-conditioned Latent Diffusion Model tailored for generating high-quality
histopathology images. Leveraging the rich contextual information provided by
pathology text reports, our approach fuses image and textual data to enhance
the generation process. By utilizing GPT's capabilities to distill and
summarize complex text reports, we establish an effective conditioning
mechanism. Through strategic conditioning and necessary architectural
enhancements, we achieved a SoTA FID score of 7.64 for text-to-image generation
on the TCGA-BRCA dataset, significantly outperforming the closest
text-conditioned competitor with FID 30.1.

本研究介绍了 PathLDM，这是一种专为生成高质量组织病理学图像而设计的首个文本条件潜在扩散模型。通过利用病理学文本报告所提供的丰富语境信息，结合图像和文本数据，通过 GPT 的能力进行复杂文本报告的提炼和总结，我们实现了在 TCGA-BRCA 数据集上文本到图像生成方面的 SoTA FID 得分为 7.64，显著优于最接近的文本条件竞争对手的 30.1 的 FID 得分。