We propose an acceleration scheme for large language models (LLMs) through
Speculative Decoding with Semantic Adaptive Tokens (SDSAT). The primary
objective of this design is to enhance the LLM model's ability to generate
draft tokens more accurately without compromising the model's accuracy. The
core strategies involve: 1) Fine-tune the model by incorporating semantic
adaptive tokens that possess flexible decoding capabilities without changing
its structure, allowing them to generate high-quality draft tokens. 2) By
employing a training method that does not affect the standard tokens, the model
can acquire parallel decoding abilities atop its original framework with
minimal training overhead. 3) We have designed the "two-step-draft-then-verify"
generation strategies using both greedy search and nucleus sampling.
Experiments conducted on the CodeLlama-13B and 7B models have yielded speed
increases of over 3.5X and 3.0X, respectively. Please refer to
this https URL

通过引入具有语义自适应能力的令牌，提出一种用于大型语言模型（LLM）的加速方案（SDSAT）。该方案通过细调模型和使用训练方法，以及使用新的生成策略，能够在不影响模型准确性的情况下，提高模型生成草稿令牌的准确性，并且在 CodeLlama-13B 和 7B 模型上实验，分别获得了超过 3.5X 和 3.0X 的加速效果。

SDSAT：通过语义自适应标记的推理进行推测解码的加速

SDSAT: Accelerating LLM Inference through Speculative Decoding with  Semantic Adaptive Tokens

Diffusion-based generative models in SE(3)-invariant space have demonstrated
promising performance in molecular conformation generation, but typically
require solving stochastic differential equations (SDEs) with thousands of
update steps. Till now, it remains unclear how to effectively accelerate this
procedure explicitly in SE(3)-invariant space, which greatly hinders its wide
application in the real world. In this paper, we systematically study the
diffusion mechanism in SE(3)-invariant space via the lens of approximate errors
induced by existing methods. Thereby, we develop more precise approximate in
SE(3) in the context of projected differential equations. Theoretical analysis
is further provided as well as empirical proof relating hyper-parameters with
such errors. Altogether, we propose a novel acceleration scheme for generating
molecular conformations in SE(3)-invariant space. Experimentally, our scheme
can generate high-quality conformations with 50x--100x speedup compared to
existing methods.

本研究通过现有方法引入的近似误差视角，系统研究了 SE (3) 不变空间中的扩散机制，从而在投影微分方程的背景下发展了更精确的近似方法，并提出了一种新的加速方案，可以在 SE (3) 不变空间中以 50 倍至 100 倍的速度生成高质量的分子构象。

加速 SE (3) 不变空间中基于扩散的分子构型生成

On Accelerating Diffusion-based Molecular Conformation Generation in  SE(3)-invariant Space

This paper studies the complexity for finding approximate stationary points
of nonconvex-strongly-concave (NC-SC) smooth minimax problems, in both general
and averaged smooth finite-sum settings. We establish nontrivial lower
complexity bounds of $\Omega(\sqrt{\kappa}\Delta L\epsilon^{-2})$ and
$\Omega(n+\sqrt{n\kappa}\Delta L\epsilon^{-2})$ for the two settings,
respectively, where $\kappa$ is the condition number, $L$ is the smoothness
constant, and $\Delta$ is the initial gap. Our result reveals substantial gaps
between these limits and best-known upper bounds in the literature. To close
these gaps, we introduce a generic acceleration scheme that deploys existing
gradient-based methods to solve a sequence of crafted
strongly-convex-strongly-concave subproblems. In the general setting, the
complexity of our proposed algorithm nearly matches the lower bound; in
particular, it removes an additional poly-logarithmic dependence on accuracy
present in previous works. In the averaged smooth finite-sum setting, our
proposed algorithm improves over previous algorithms by providing a
nearly-tight dependence on the condition number.

本文研究非凸强凹（NC-SC）平滑极小值问题的近似稳定点的复杂度，在一般和平均平滑有限和设置中建立了非平凡的较低复杂度下界。我们提出了一种通用的加速方案，使用现有的基于梯度的方法来解决一系列的精心制作的强凸 - 强凹子问题，进而缩小了复杂度差距，尤其是在一般情况下，我们提出的算法的复杂度几乎与下界相当。