Inspired by the software industry's practice of offering different editions
or versions of a product tailored to specific user groups or use cases, we
propose a novel task, namely, training-free editioning, for text-to-image
models. Specifically, we aim to create variations of a base text-to-image model
without retraining, enabling the model to cater to the diverse needs of
different user groups or to offer distinct features and functionalities. To
achieve this, we propose that different editions of a given text-to-image model
can be formulated as concept subspaces in the latent space of its text encoder
(e.g., CLIP). In such a concept subspace, all points satisfy a specific user
need (e.g., generating images of a cat lying on the grass/ground/falling
leaves). Technically, we apply Principal Component Analysis (PCA) to obtain the
desired concept subspaces from representative text embedding that correspond to
a specific user need or requirement. Projecting the text embedding of a given
prompt into these low-dimensional subspaces enables efficient model editioning
without retraining. Intuitively, our proposed editioning paradigm enables a
service provider to customize the base model into its "cat edition" (or other
editions) that restricts image generation to cats, regardless of the user's
prompt (e.g., dogs, people, etc.). This introduces a new dimension for product
differentiation, targeted functionality, and pricing strategies, unlocking
novel business models for text-to-image generators. Extensive experimental
results demonstrate the validity of our approach and its potential to enable a
wide range of customized text-to-image model editions across various domains
and applications.

通过在文本编码器的潜在空间中形成概念子空间，我们提出了一种新颖的任务，即无需重新训练即可创建基于特定用户需求的变体，并且通过在这些低维度子空间中投影给定提示的文本嵌入，实现高效的模型定制。这一提议为文本到图像生成器的定制版开启了新的产品差异化、目标功能和定价策略维度，为各个领域和应用开展了一系列定制化的文本到图像模型版本。

无需训练的文本到图像模型编辑

Training-free Editioning of Text-to-Image Models

Concepts are key building blocks of higher level human understanding.
Explainable AI (XAI) methods have shown tremendous progress in recent years,
however, local attribution methods do not allow to identify coherent model
behavior across samples and therefore miss this essential component. In this
work, we study concept-based explanations and put forward a new definition of
concepts as low-dimensional subspaces of hidden feature layers. We novelly
apply sparse subspace clustering to discover these concept subspaces. Moving
forward, we derive insights from concept subspaces in terms of localized input
(concept) maps, show how to quantify concept relevances and lastly, evaluate
similarities and transferability between concepts. We empirically demonstrate
the soundness of the proposed Sparse Subspace Clustering for Concept Discovery
(SSCCD) method for a variety of different image classification tasks. This
approach allows for deeper insights into the actual model behavior that would
remain hidden from conventional input-level heatmaps.

通过稀疏子空间聚类的方法，探究概念基础解释和概念子空间作为深度学习模型行为的关键组成部分，从而更深入地理解模型行为。