Diffusion models have recently gained unprecedented attention in the field of
image synthesis due to their remarkable generative capabilities.
Notwithstanding their prowess, these models often incur substantial
computational costs, primarily attributed to the sequential denoising process
and cumbersome model size. Traditional methods for compressing diffusion models
typically involve extensive retraining, presenting cost and feasibility
challenges. In this paper, we introduce DeepCache, a novel training-free
paradigm that accelerates diffusion models from the perspective of model
architecture. DeepCache capitalizes on the inherent temporal redundancy
observed in the sequential denoising steps of diffusion models, which caches
and retrieves features across adjacent denoising stages, thereby curtailing
redundant computations. Utilizing the property of the U-Net, we reuse the
high-level features while updating the low-level features in a very cheap way.
This innovative strategy, in turn, enables a speedup factor of 2.3$\times$ for
Stable Diffusion v1.5 with only a 0.05 decline in CLIP Score, and 4.1$\times$
for LDM-4-G with a slight decrease of 0.22 in FID on ImageNet. Our experiments
also demonstrate DeepCache's superiority over existing pruning and distillation
methods that necessitate retraining and its compatibility with current sampling
techniques. Furthermore, we find that under the same throughput, DeepCache
effectively achieves comparable or even marginally improved results with DDIM
or PLMS. The code is available at this https URL

Diffusion models have gained attention in image synthesis, and this paper introduces DeepCache, a training-free paradigm that accelerates diffusion models by capitalizing on temporal redundancy in denoising steps and achieving a speedup factor of 2.3x for Stable Diffusion v1.5 and 4.1x for LDM-4-G without significant decline in CLIP Score or FID on ImageNet.

DeepCache：加速扩散模型的自由之选

DeepCache: Accelerating Diffusion Models for Free

We present DeepCache, a principled cache design for deep learning inference
in continuous mobile vision. DeepCache benefits model execution efficiency by
exploiting temporal locality in input video streams. It addresses a key
challenge raised by mobile vision: the cache must operate under video scene
variation, while trading off among cacheability, overhead, and loss in model
accuracy. At the input of a model, DeepCache discovers video temporal locality
by exploiting the video's internal structure, for which it borrows proven
heuristics from video compression; into the model, DeepCache propagates regions
of reusable results by exploiting the model's internal structure. Notably,
DeepCache eschews applying video heuristics to model internals which are not
pixels but high-dimensional, difficult-to-interpret data. Our implementation of
DeepCache works with unmodified deep learning models, requires zero developer's
manual effort, and is therefore immediately deployable on off-the-shelf mobile
devices. Our experiments show that DeepCache saves inference execution time by
18% on average and up to 47%. DeepCache reduces system energy consumption by
20% on average.

DeepCache 是一种基于时间局部性的缓存设计，能提高深度学习推理的效率且适用于移动端应用，能平衡缓存能力、资源开销和模型精度损失等问题，并在实验中展现了一定的推理时间和系统能耗的优化。