Large language models have shown impressive capabilities across a variety of
NLP tasks, yet their generating text autoregressively is time-consuming. One
way to speed them up is speculative decoding, which generates candidate
segments (a sequence of tokens) from a fast draft model that is then verified
in parallel by the target model. However, the acceptance rate of candidate
tokens receives limitations from several factors, such as the model, the
dataset, and the decoding setup. This paper proposes sampling multiple
candidates from a draft model and then organising them in batches for
verification. We design algorithms for efficient multi-candidate verification
while maintaining the distribution of the target model. Our approach shows
significant improvements in acceptance rates on multiple datasets and models,
consistently outperforming standard speculative decoding.

大型语言模型在各种自然语言处理任务中展现出令人印象深刻的能力，但是自回归生成文本却非常耗时。提高速度的一个方法是进行猜测性解码，即由快速的草稿模型生成候选分段（一系列令牌），然后由目标模型并行验证。然而，候选标记的接受率受到模型、数据集和解码设置等多个因素的限制。本文提出了从草稿模型中采样多个候选分段，并将它们分批进行验证的方法。我们设计了高效的多候选验证算法，同时保持目标模型的分布。我们的方法在多个数据集和模型上都展现出显著的接受率改进，始终优于标准的猜测性解码。

多候选猜测解码

Multi-Candidate Speculative Decoding

Methods of approximate Bayesian computation (ABC) are increasingly used for
analysis of complex models. A major challenge for ABC is over-coming the often
inherent problem of high rejection rates in the accept/reject methods based on
prior:predictive sampling. A number of recent developments aim to address this
with extensions based on sequential Monte Carlo (SMC) strategies. We build on
this here, introducing an ABC SMC method that uses data-based adaptive weights.
This easily implemented and computationally trivial extension of ABC SMC can
very substantially improve acceptance rates, as is demonstrated in a series of
examples with simulated and real data sets, including a currently topical
example from dynamic modelling in systems biology applications.

介绍了一种采用基于数据的自适应权重的 ABC SMC 方法，可在复杂模型中实现更高的接受率，适用于动态建模等系统生物应用。