With the advance of diffusion models, various personalized image generation
methods have been proposed. However, almost all existing work only focuses on
either subject-driven or style-driven personalization. Meanwhile,
state-of-the-art methods face several challenges in realizing compositional
personalization, i.e., composing different subject and style concepts, such as
concept disentanglement, unified reconstruction paradigm, and insufficient
training data. To address these issues, we introduce FreeTuner, a flexible and
training-free method for compositional personalization that can generate any
user-provided subject in any user-provided style (see Figure 1). Our approach
employs a disentanglement strategy that separates the generation process into
two stages to effectively mitigate concept entanglement. FreeTuner leverages
the intermediate features within the diffusion model for subject concept
representation and introduces style guidance to align the synthesized images
with the style concept, ensuring the preservation of both the subject's
structure and the style's aesthetic features. Extensive experiments have
demonstrated the generation ability of FreeTuner across various personalization
settings.

FreeTuner 是一种灵活的、无需训练的组合个性化方法，可以以任意用户提供的主题和风格生成图像，通过分解生成过程为两个阶段来有效缓解概念纠缠，利用扩散模型中的中间特征进行主题概念表达，并引入风格指导以确保生成的图像既保留了主题结构又呈现了风格的美感特征，通过大量实验证明了 FreeTuner 在各种个性化设置下的生成能力。

FreeTuner：无需训练的扩散式任意主题和风格

FreeTuner: Any Subject in Any Style with Training-free Diffusion

Explaining deep learning model inferences is a promising venue for scientific
understanding, improving safety, uncovering hidden biases, evaluating fairness,
and beyond, as argued by many scholars. One of the principal benefits of
counterfactual explanations is allowing users to explore "what-if" scenarios
through what does not and cannot exist in the data, a quality that many other
forms of explanation such as heatmaps and influence functions are inherently
incapable of doing. However, most previous work on generative explainability
cannot disentangle important concepts effectively, produces unrealistic
examples, or fails to retain relevant information. We propose a novel approach,
DISSECT, that jointly trains a generator, a discriminator, and a concept
disentangler to overcome such challenges using little supervision. DISSECT
generates Concept Traversals (CTs), defined as a sequence of generated examples
with increasing degrees of concepts that influence a classifier's decision. By
training a generative model from a classifier's signal, DISSECT offers a way to
discover a classifier's inherent "notion" of distinct concepts automatically
rather than rely on user-predefined concepts. We show that DISSECT produces CTs
that (1) disentangle several concepts, (2) are influential to a classifier's
decision and are coupled to its reasoning due to joint training (3), are
realistic, (4) preserve relevant information, and (5) are stable across similar
inputs. We validate DISSECT on several challenging synthetic and realistic
datasets where previous methods fall short of satisfying desirable criteria for
interpretability and show that it performs consistently well and better than
existing methods. Finally, we present experiments showing applications of
DISSECT for detecting potential biases of a classifier and identifying spurious
artifacts that impact predictions.

这篇论文提出了一种名为 DISSECT 的新方法，该方法可以有效地实现生成解释、概念解缠、反事实解释、可解释性和偏见检测等课题。该方法使用少量监督同时训练生成器、鉴别器和概念解缠器，通过生成分类器辨别信号的生成模型，能够自动发现分类器固有的不同概念。应用该方法能够检测分类器的潜在偏见和识别影响预测的虚假因素。