Speculative decoding is a relatively new decoding framework that leverages
small and efficient draft models to reduce the latency of LLMs. In this study,
we introduce GliDe and CaPE, two low-hassle modifications to vanilla
speculative decoding to further improve the decoding speed of a frozen LLM.
Specifically, GliDe is a modified draft model architecture that reuses the
cached keys and values from the target LLM, while CaPE is a proposal expansion
method that uses the draft model's confidence scores to help select additional
candidate tokens for verification. Extensive experiments on different
benchmarks demonstrate that our proposed GliDe draft model significantly
reduces the expected decoding latency. Additional evaluation using walltime
reveals that GliDe can accelerate Vicuna models up to 2.17x and further extend
the improvement to 2.61x with CaPE. We will release our code, data, and the
trained draft models.

该研究介绍了两种改进的解码模型，GliDe 和 CaPE，以进一步提高冻结 LLM 的解码速度，并通过大量实验证明 GliDe 可以显著降低期望的解码延迟。

佩戴披风的 GliDe：一种简便的加速推测式解码方法

GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative  Decoding

Diffusion models have garnered significant attention since they can
effectively learn complex multivariate Gaussian distributions, resulting in
diverse, high-quality outcomes. They introduce Gaussian noise into training
data and reconstruct the original data iteratively. Central to this iterative
process is a single Unet, adapting across time steps to facilitate generation.
Recent work revealed the presence of composition and denoising phases in this
generation process, raising questions about the Unets' varying roles. Our study
dives into the dynamic behavior of Unets within denoising diffusion
probabilistic models (DDPM), focusing on (de)convolutional blocks and skip
connections across time steps. We propose an analytical method to
systematically assess the impact of time steps and core Unet components on the
final output. This method eliminates components to study causal relations and
investigate their influence on output changes. The main purpose is to
understand the temporal dynamics and identify potential shortcuts during
inference. Our findings provide valuable insights into the various generation
phases during inference and shed light on the Unets' usage patterns across
these phases. Leveraging these insights, we identify redundancies in GLIDE (an
improved DDPM) and improve inference time by ~27% with minimal degradation in
output quality. Our ultimate goal is to guide more informed optimization
strategies for inference and influence new model designs.

通过研究 Unet 在去噪扩散概率模型中的动态行为，我们系统评估了时间步长和核心 Unet 组件对最终输出的影响，发现其中的生成阶段和 Unet 在推断中的使用模式，进而识别出改进 DDPM 中的冗余以提高推断速度，同时在输出质量方面几乎没有损失。我们的目标是为推断过程指导更明智的优化策略并影响新模型设计。

解构扩散模型中 Unet 的时态动力学

Unraveling the Temporal Dynamics of the Unet in Diffusion Models

Digital art restoration has benefited from inpainting models to correct the
degradation or missing sections of a painting. This work compares three current
state-of-the art models for inpainting of large missing regions. We provide
qualitative and quantitative comparison of the performance by CoModGANs, LaMa
and GLIDE in inpainting of blurry and missing sections of images. We use
Escher's incomplete painting Print Gallery as our test study since it presents
several of the challenges commonly present in restorative inpainting.

本文研究了数字艺术修复领域中三种最先进的新兴的大面积修复模型 CoModGANs，LaMa 和 GLIDE。通过我们的测试研究 - 艾舍的不完整的画作 “图片展”，我们提供了这些模型在图像的模糊和丢失部分修复方面的定量和定性比较。