Diffusion models currently dominate the field of data-driven image synthesis
with their unparalleled scaling to large datasets. In this paper, we identify
and rectify several causes for uneven and ineffective training in the popular
ADM diffusion model architecture, without altering its high-level structure.
Observing uncontrolled magnitude changes and imbalances in both the network
activations and weights over the course of training, we redesign the network
layers to preserve activation, weight, and update magnitudes on expectation. We
find that systematic application of this philosophy eliminates the observed
drifts and imbalances, resulting in considerably better networks at equal
computational complexity. Our modifications improve the previous record FID of
2.41 in ImageNet-512 synthesis to 1.81, achieved using fast deterministic
sampling.
As an independent contribution, we present a method for setting the
exponential moving average (EMA) parameters post-hoc, i.e., after completing
the training run. This allows precise tuning of EMA length without the cost of
performing several training runs, and reveals its surprising interactions with
network architecture, training time, and guidance.

该研究提出了改进 ADM 扩散模型架构以解决不均匀训练、网络激活以及指数移动平均参数的问题，使得图像合成在计算复杂度相等的情况下得到了显著改进。

分析和改进扩散模型的训练动力学

Analyzing and Improving the Training Dynamics of Diffusion Models

We develop three efficient approaches for generating visual explanations from
3D convolutional neural networks (3D-CNNs) for Alzheimer's disease
classification. One approach conducts sensitivity analysis on hierarchical 3D
image segmentation, and the other two visualize network activations on a
spatial map. Visual checks and a quantitative localization benchmark indicate
that all approaches identify important brain parts for Alzheimer's disease
diagnosis. Comparative analysis show that the sensitivity analysis based
approach has difficulty handling loosely distributed cerebral cortex, and
approaches based on visualization of activations are constrained by the
resolution of the convolutional layer. The complementarity of these methods
improves the understanding of 3D-CNNs in Alzheimer's disease classification
from different perspectives.

研究开发了三种有效的方法来从三维卷积神经网络 (3D-CNN) 中生成视觉解释，用于阿尔茨海默病分类。这些方法基于敏感性分析和网络激活的可视化，并且都能够识别出对阿尔茨海默病诊断非常重要的脑部部位，并提高了对 3D-CNN 在阿尔茨海默病分类中的理解。