Text-to-image diffusion models have made significant progress in generating
naturalistic images from textual inputs, and demonstrate the capacity to learn
and represent complex visual-semantic relationships. While these diffusion
models have achieved remarkable success, the underlying mechanisms driving
their performance are not yet fully accounted for, with many unanswered
questions surrounding what they learn, how they represent visual-semantic
relationships, and why they sometimes fail to generalize. Our work presents
Diffusion Partial Information Decomposition (DiffusionPID), a novel technique
that applies information-theoretic principles to decompose the input text
prompt into its elementary components, enabling a detailed examination of how
individual tokens and their interactions shape the generated image. We
introduce a formal approach to analyze the uniqueness, redundancy, and synergy
terms by applying PID to the denoising model at both the image and pixel level.
This approach enables us to characterize how individual tokens and their
interactions affect the model output. We first present a fine-grained analysis
of characteristics utilized by the model to uniquely localize specific
concepts, we then apply our approach in bias analysis and show it can recover
gender and ethnicity biases. Finally, we use our method to visually
characterize word ambiguity and similarity from the model's perspective and
illustrate the efficacy of our method for prompt intervention. Our results show
that PID is a potent tool for evaluating and diagnosing text-to-image diffusion
models.

Diffusion Partial Information Decomposition (DiffusionPID) is a novel technique applied to decompose the input text prompt, enabling a detailed examination of how individual tokens and their interactions shape the generated image in text-to-image diffusion models, addressing unanswered questions and biases analysis.