The success of the text-guided diffusion model has inspired the development
and release of numerous powerful diffusion models within the open-source
community. These models are typically fine-tuned on various expert datasets,
showcasing diverse denoising capabilities. Leveraging multiple high-quality
models to produce stronger generation ability is valuable, but has not been
extensively studied. Existing methods primarily adopt parameter merging
strategies to produce a new static model. However, they overlook the fact that
the divergent denoising capabilities of the models may dynamically change
across different states, such as when experiencing different prompts, initial
noises, denoising steps, and spatial locations. In this paper, we propose a
novel ensembling method, Adaptive Feature Aggregation (AFA), which dynamically
adjusts the contributions of multiple models at the feature level according to
various states (i.e., prompts, initial noises, denoising steps, and spatial
locations), thereby keeping the advantages of multiple diffusion models, while
suppressing their disadvantages. Specifically, we design a lightweight
Spatial-Aware Block-Wise (SABW) feature aggregator that adaptive aggregates the
block-wise intermediate features from multiple U-Net denoisers into a unified
one. The core idea lies in dynamically producing an individual attention map
for each model's features by comprehensively considering various states. It is
worth noting that only SABW is trainable with about 50 million parameters,
while other models are frozen. Both the quantitative and qualitative
experiments demonstrate the effectiveness of our proposed Adaptive Feature
Aggregation method. The code is available at this https URL

提出了一种新的集成方法，自适应特征聚合（AFA），它根据各种状态（即提示、初始噪声、去噪步骤和空间位置）动态调整多个模型在特征级别的贡献，从而保持多个扩散模型的优势，并抑制它们的缺点。

通过自适应特征聚合集成扩散模型

Ensembling Diffusion Models via Adaptive Feature Aggregation

The loss functions of deep neural networks are complex and their geometric
properties are not well understood. We show that the optima of these complex
loss functions are in fact connected by simple curves over which training and
test accuracy are nearly constant. We introduce a training procedure to
discover these high-accuracy pathways between modes. Inspired by this new
geometric insight, we also propose a new ensembling method entitled Fast
Geometric Ensembling (FGE). Using FGE we can train high-performing ensembles in
the time required to train a single model. We achieve improved performance
compared to the recent state-of-the-art Snapshot Ensembles, on CIFAR-10,
CIFAR-100, and ImageNet.

本研究利用一种新的几何洞见发现了深度神经网络损失函数的凸性及优化过程中损失函数可达到极值的简单曲线，提出了一种利用此洞见的训练方法，并提出了一种名为快速几何融合（FGE）的新的集成方法，在 CIFAR-10、CIFAR-100 和 ImageNet 三个数据集上，与最新的快照集成法相比取得了更好的性能。