Diffusion models are a powerful generative framework, but come with expensive
inference. Existing acceleration methods often compromise image quality or fail
under complex conditioning when operating in an extremely low-step regime. In
this work, we propose a novel distillation framework tailored to enable
high-fidelity, diverse sample generation using just one to three steps. Our
approach comprises three key components: (i) Backward Distillation, which
mitigates training-inference discrepancies by calibrating the student on its
own backward trajectory; (ii) Shifted Reconstruction Loss that dynamically
adapts knowledge transfer based on the current time step; and (iii) Noise
Correction, an inference-time technique that enhances sample quality by
addressing singularities in noise prediction. Through extensive experiments, we
demonstrate that our method outperforms existing competitors in quantitative
metrics and human evaluations. Remarkably, it achieves performance comparable
to the teacher model using only three denoising steps, enabling efficient
high-quality generation.

通过反向蒸馏、动态适应的位移重构损失和噪声修正技术，我们提出了一种新的蒸馏框架，用于实现高保真度、多样化的样本生成，仅使用一到三个步骤即可。通过大量实验证明，我们的方法在定量指标和人工评估中优于现有竞争对手，仅仅使用三个降噪步骤即可达到与教师模型相当的性能，从而实现了高效高质量的生成。

想象 Flash：使用向后蒸馏加速鸵鸟扩散模型

Imagine Flash: Accelerating Emu Diffusion Models with Backward  Distillation

Unsupervised Domain Adaptation (UDA) aims to solve the problem of label
scarcity of the target domain by transferring the knowledge from the label rich
source domain. Usually, the source domain consists of synthetic images for
which the annotation is easily obtained using the well known computer graphics
techniques. However, obtaining annotation for real world images (target domain)
require lot of manual annotation effort and is very time consuming because it
requires per pixel annotation. To address this problem we propose SegDA module
to enhance transfer performance of UDA methods by learning the maximum
separable segment representation. This resolves the problem of identifying
visually similar classes like pedestrian/rider, sidewalk/road etc. We leveraged
Equiangular Tight Frame (ETF) classifier inspired from Neural Collapse for
maximal separation between segment classes. This causes the source domain pixel
representation to collapse to a single vector forming a simplex vertices which
are aligned to the maximal separable ETF classifier. We use this phenomenon to
propose the novel architecture for domain adaptation of segment representation
for target domain. Additionally, we proposed to estimate the noise in labelling
the target domain images and update the decoder for noise correction which
encourages the discovery of pixels for classes not identified in pseudo labels.
We have used four UDA benchmarks simulating synthetic-to-real,
daytime-to-nighttime, clear-to-adverse weather scenarios. Our proposed approach
outperforms +2.2 mIoU on GTA -> Cityscapes, +2.0 mIoU on Synthia -> Cityscapes,
+5.9 mIoU on Cityscapes -> DarkZurich, +2.6 mIoU on Cityscapes -> ACDC.

通过使用 SynthDA 模块、Equiangular Tight Frame (ETF) 分类器以及噪声校正等方法，我们的研究论文在四个不同的领域适应性基准测试上都取得了显著的改进。