Diffusion models are the state of the art in text-to-image generation, but
their perceptual variability remains understudied. In this paper, we examine
how prompts affect image variability in black-box diffusion-based models. We
propose W1KP, a human-calibrated measure of variability in a set of images,
bootstrapped from existing image-pair perceptual distances. Current datasets do
not cover recent diffusion models, thus we curate three test sets for
evaluation. Our best perceptual distance outperforms nine baselines by up to 18
points in accuracy, and our calibration matches graded human judgements 78% of
the time. Using W1KP, we study prompt reusability and show that Imagen prompts
can be reused for 10-50 random seeds before new images become too similar to
already generated images, while Stable Diffusion XL and DALL-E 3 can be reused
50-200 times. Lastly, we analyze 56 linguistic features of real prompts,
finding that the prompt's length, CLIP embedding norm, concreteness, and word
senses influence variability most. As far as we are aware, we are the first to
analyze diffusion variability from a visuolinguistic perspective. Our project
page is at this http URL

基于扩散模型的文本到图像生成是现阶段的最先进技术，本研究通过对提示语的影响来探究黑盒扩散模型中的图像可变性，提出了 W1KP 人工校准的图像可变性度量方法，评估了新型扩散模型的性能。发现基于 W1KP 方法，在精确性方面胜过其他九个基线模型最高达 18 个点，且人工校准结果与人类判断 78% 的一致性。同时，利用 W1KP，研究了提示语的可重用性，表明 Imagen 提示语可重复利用 10-50 次，Stable Diffusion XL 和 DALL-E3 可以重复利用 50-200 次。最后，通过分析真实提示语的 56 个语言特征，发现提示语的长度、CLIP 嵌入向量范数、具象度和词义影响图像的可变性。据我们所知，本研究是首个从视觉语言角度分析扩散可变性的研究。详细信息请参考项目页面：http URL

文字胜过千言万语：衡量和理解文本到图像生成中的知觉变异性

Words Worth a Thousand Pictures: Measuring and Understanding Perceptual  Variability in Text-to-Image Generation

Despite the recent success of deep learning methods at achieving new
state-of-the-art accuracy for medical image segmentation, some major
limitations are still restricting their deployment into clinics. One major
limitation of deep learning-based segmentation methods is their lack of
robustness to variability in the image acquisition protocol and in the imaged
anatomy that were not represented or were underrepresented in the training
dataset. This suggests adding new manually segmented images to the training
dataset to better cover the image variability. However, in most cases, the
manual segmentation of medical images requires highly skilled raters and is
time-consuming, making this solution prohibitively expensive. Even when
manually segmented images from different sources are available, they are rarely
annotated for exactly the same regions of interest. This poses an additional
challenge for current state-of-the-art deep learning segmentation methods that
rely on supervised learning and therefore require all the regions of interest
to be segmented for all the images to be used for training. This thesis
introduces new mathematical and optimization methods to mitigate those
limitations.

该论文研究利用新的数学和优化方法来解决基于深度学习的医学图像分割方法在图像获取协议和图像解剖变异性方面的限制，以减少需要手动标记成本高昂的问题。