Recent advancements in diffusion models have positioned them at the forefront
of image generation. Despite their superior performance, diffusion models are
not without drawbacks; they are characterized by complex architectures and
substantial computational demands, resulting in significant latency due to
their iterative sampling process. To mitigate these limitations, we introduce a
dual approach involving model miniaturization and a reduction in sampling
steps, aimed at significantly decreasing model latency. Our methodology
leverages knowledge distillation to streamline the U-Net and image decoder
architectures, and introduces an innovative one-step DM training technique that
utilizes feature matching and score distillation. We present two models,
SDXS-512 and SDXS-1024, achieving inference speeds of approximately 100 FPS
(30x faster than SD v1.5) and 30 FP (60x faster than SDXL) on a single GPU,
respectively. Moreover, our training approach offers promising applications in
image-conditioned control, facilitating efficient image-to-image translation.

通过模型微型化、减少采样步骤和知识蒸馏的双重方法，减小模型延迟，本研究提出了一种在图像生成领域优化扩散模型性能的方法。通过简化 U-Net 和图像解码器的结构并引入一步式 DM 训练技术，利用特征匹配和得分蒸馏，我们实现了两个模型 SDXS-512 和 SDXS-1024，在单个 GPU 上实现了约 100 帧每秒（比 SD v1.5 快 30 倍）和 30 帧每秒（比 SDXL 快 60 倍）的推理速度。此外，我们的训练方法还在图像条件控制方面具有潜在的应用，可以实现高效的图像转换。

SDXS：带图像条件的实时单步潜变扩散模型

SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions

Current state-of-the-art object detectors are at the expense of high
computational costs and are hard to deploy to low-end devices. Knowledge
distillation, which aims at training a smaller student network by transferring
knowledge from a larger teacher model, is one of the promising solutions for
model miniaturization. In this paper, we investigate each module of a typical
detector in depth, and propose a general distillation framework that adaptively
transfers knowledge from teacher to student according to the task specific
priors. The intuition is that simply distilling all information from teacher to
student is not advisable, instead we should only borrow priors from the teacher
model where the student cannot perform well. Towards this goal, we propose a
region proposal sharing mechanism to interflow region responses between the
teacher and student models. Based on this, we adaptively transfer knowledge at
three levels, \emph{i.e.}, feature backbone, classification head, and bounding
box regression head, according to which model performs more reasonably.
Furthermore, considering that it would introduce optimization dilemma when
minimizing distillation loss and detection loss simultaneously, we propose a
distillation decay strategy to help improve model generalization via gradually
reducing the distillation penalty. Experiments on widely used detection
benchmarks demonstrate the effectiveness of our method. In particular, using
Faster R-CNN with FPN as an instantiation, we achieve an accuracy of $39.0\%$
with Resnet-50 on COCO dataset, which surpasses the baseline $36.3\%$ by
$2.7\%$ points, and even better than the teacher model with $38.5\%$ mAP.

该论文提出了一种基于知识蒸馏的对象检测模型微调方法，采用区域提议共享机制和自适应知识转移方法，根据模型的不同表现区分性能好坏，同时使用知识衰减策略帮助提高模型的泛化性能。实验结果表明，该方法在 COOC 数据集上取得了比基线模型更好的检测结果。