Diffusion models achieve great success in generating diverse and
high-fidelity images. The performance improvements come with low generation
speed per image, which hinders the application diffusion models in real-time
scenarios. While some certain predictions benefit from the full computation of
the model in each sample iteration, not every iteration requires the same
amount of computation, potentially leading to computation waste. In this work,
we propose DeeDiff, an early exiting framework that adaptively allocates
computation resources in each sampling step to improve the generation
efficiency of diffusion models. Specifically, we introduce a timestep-aware
uncertainty estimation module (UEM) for diffusion models which is attached to
each intermediate layer to estimate the prediction uncertainty of each layer.
The uncertainty is regarded as the signal to decide if the inference
terminates. Moreover, we propose uncertainty-aware layer-wise loss to fill the
performance gap between full models and early-exited models. With such loss
strategy, our model is able to obtain comparable results as full-layer models.
Extensive experiments of class-conditional, unconditional, and text-guided
generation on several datasets show that our method achieves state-of-the-art
performance and efficiency trade-off compared with existing early exiting
methods on diffusion models. More importantly, our method even brings extra
benefits to baseline models and obtains better performance on CIFAR-10 and
Celeb-A datasets. Full code and model are released for reproduction.

我们提出了 DeeDiff，这是一个适应性地分配计算资源的早期退出框架，用于提高扩散模型的生成效率。通过引入基于时间步长的不确定性估计模块（UEM），我们的模型能够根据不同层的预测不确定性来决定推断的终止。通过使用基于不确定性的逐层损失策略，我们的模型能够获得与全模型相媲美的结果。在多个数据集上进行的实验表明，我们的方法在扩散模型上实现了最先进的性能和效率平衡，并且对于 CIFAR-10 和 Celeb-A 数据集甚至带来了额外的收益。

DeeDiff: 动态不确定性感知的早期退出加速扩散模型生成

DeeDiff: Dynamic Uncertainty-Aware Early Exiting for Accelerating  Diffusion Model Generation

The sequence length along the time axis is often the dominant factor of the
computation in speech processing. Works have been proposed to reduce the
sequence length for lowering the computational cost in self-supervised speech
models. However, different downstream tasks have different tolerance of
sequence compressing, so a model that produces a fixed compressing rate may not
fit all tasks. In this work, we introduce a once-for-all (OFA) sequence
compression framework for self-supervised speech models that supports a
continuous range of operating compressing rates. The framework is evaluated on
various tasks, showing marginal degradation compared to the fixed compressing
rate variants with a smooth performance-efficiency trade-off. We further
explore adaptive compressing rate learning, demonstrating the ability to select
task-specific preferred frame periods without needing a grid search.

本研究提出了一种可以在多种下游任务中调节压缩率的模型压缩方法，该方法可以在保证语音模型性能的前提下，减小计算成本并且不需要通过网格搜索来确定不同任务的压缩率。