The QUILT-1M dataset is the first openly available dataset containing images
harvested from various online sources. While it provides a huge data variety,
the image quality and composition is highly heterogeneous, impacting its
utility for text-conditional image synthesis. We propose an automatic pipeline
that provides predictions of the most common impurities within the images,
e.g., visibility of narrators, desktop environment and pathology software, or
text within the image. Additionally, we propose to use semantic alignment
filtering of the image-text pairs. Our findings demonstrate that by rigorously
filtering the dataset, there is a substantial enhancement of image fidelity in
text-to-image tasks.

通过使用自动化流程和语义对齐筛选图像与文本对，我们的研究发现从 QUILT-1M 数据集中滤除常见杂质可显著提高文本到图像任务中的图像保真度。

基于模型的 QUILT-1M 病理学数据集清洗用于文本条件图像合成

Model-based Cleaning of the QUILT-1M Pathology Dataset for  Text-Conditional Image Synthesis

Diffusion models have recently been shown to generate high-quality synthetic
images, especially when paired with a guidance technique to trade off diversity
for fidelity. We explore diffusion models for the problem of text-conditional
image synthesis and compare two different guidance strategies: CLIP guidance
and classifier-free guidance. We find that the latter is preferred by human
evaluators for both photorealism and caption similarity, and often produces
photorealistic samples. Samples from a 3.5 billion parameter text-conditional
diffusion model using classifier-free guidance are favored by human evaluators
to those from DALL-E, even when the latter uses expensive CLIP reranking.
Additionally, we find that our models can be fine-tuned to perform image
inpainting, enabling powerful text-driven image editing. We train a smaller
model on a filtered dataset and release the code and weights at
this https URL

本研究探讨了扩散模型在文本条件下生成图像的问题，并比较了不同的指导策略：CLIP 指导和无分类器指导。 结果发现对于照片逼真度和字幕相似性，后者更受人类评估人员的青睐，还可以进行图像修复。