Deployment of autoregressive large language models (LLMs) is costly, and as
these models increase in size, the associated costs will become even more
considerable. Consequently, different methods have been proposed to accelerate
the token generation process and reduce costs. Speculative decoding (SD) is
among the most promising approaches to speed up the LLM decoding process by
verifying multiple tokens in parallel and using an auxiliary smaller draft
model to generate the possible tokens. In SD, usually, one draft model is used
to serve a specific target model; however, in practice, LLMs are diverse, and
we might need to deal with many target models or more than one target model
simultaneously. In this scenario, it is not clear which draft model should be
used for which target model, and searching among different draft models or
training customized draft models can further increase deployment costs. In this
paper, we first introduce a novel multi-target scenario for the deployment of
draft models for faster inference. Then, we present a novel, more efficient
sorted speculative decoding mechanism that outperforms regular baselines in
multi-target settings. We evaluated our method on Spec-Bench in different
settings, including base models such as Vicuna 7B, 13B, and LLama Chat 70B. Our
results suggest that our draft models perform better than baselines for
multiple target models at the same time.

部署异步大型语言模型（LLMs）的代价高昂，为了减少成本，本文介绍了一种新的多目标场景的 Draft 模型部署方法，并提出了一种更高效的排序 Speculative Decoding 机制，该方法在多目标环境下优于基线模型。

S2D：排序的推测解码，用于更高效部署嵌套大型语言模型

S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested  Large Language Models

Recently, voice conversion (VC) without parallel data has been successfully
adapted to multi-target scenario in which a single model is trained to convert
the input voice to many different speakers. However, such model suffers from
the limitation that it can only convert the voice to the speakers in the
training data, which narrows down the applicable scenario of VC. In this paper,
we proposed a novel one-shot VC approach which is able to perform VC by only an
example utterance from source and target speaker respectively, and the source
and target speaker do not even need to be seen during training. This is
achieved by disentangling speaker and content representations with instance
normalization (IN). Objective and subjective evaluation shows that our model is
able to generate the voice similar to target speaker. In addition to the
performance measurement, we also demonstrate that this model is able to learn
meaningful speaker representations without any supervision.

本文提出了一种使用仅一次源和目标说话者示例语音，通过实例归一化来分解说话者和语音内容表示的一次性语音转换方法，无需事先见过源和目标说话者训练模型即可执行声音转换，目标语音及说话者的相似性得到了客观和主观的评估，同时表明该模型可以在没有任何监督的情况下学习含义深层次的说话者表示。