This technical report describes the design and training of novel speculative
decoding draft models, for accelerating the inference speeds of large language
models in a production environment. By conditioning draft predictions on both
context vectors and sampled tokens, we can train our speculators to efficiently
predict high-quality n-grams, which the base model then accepts or rejects.
This allows us to effectively predict multiple tokens per inference forward
pass, accelerating wall-clock inference speeds of highly optimized base model
implementations by a factor of 2-3x. We explore these initial results and
describe next steps for further improvements.

设计和训练新型的推测解码草稿模型，通过在上下文向量和样本标记上条件化草稿预测，可以高效预测高质量的 n-gram，以提高大型语言模型在生产环境中的推理速度。

使用混合标记 / 嵌入推测器加速生产 LLM

Accelerating Production LLMs with Combined Token/Embedding Speculators

Dataset augmentation, the practice of applying a wide array of
domain-specific transformations to synthetically expand a training set, is a
standard tool in supervised learning. While effective in tasks such as visual
recognition, the set of transformations must be carefully designed,
implemented, and tested for every new domain, limiting its re-use and
generality. In this paper, we adopt a simpler, domain-agnostic approach to
dataset augmentation. We start with existing data points and apply simple
transformations such as adding noise, interpolating, or extrapolating between
them. Our main insight is to perform the transformation not in input space, but
in a learned feature space. A re-kindling of interest in unsupervised
representation learning makes this technique timely and more effective. It is a
simple proposal, but to-date one that has not been tested empirically. Working
in the space of context vectors generated by sequence-to-sequence models, we
demonstrate a technique that is effective for both static and sequential data.

该研究提出了一种基于简单转换的数据集增强方法，通过在已有数据点上进行特征空间的变换，从而促进监督和无监督学习技术在静态和序列数据上的应用。