Many language generation models are now available for a wide range of
generation tasks, including machine translation and summarization. Combining
such diverse models may lead to further progress, but ensembling generation
models is challenging during inference: conventional ensembling methods (e.g.,
shallow fusion) require that the models share vocabulary/tokenization schemes.
We introduce Twist decoding, a simple and general text generation algorithm
that benefits from diverse models at inference time. Our method does not assume
the vocabulary, tokenization or even generation order is shared. Our extensive
evaluations on machine translation and scientific paper summarization
demonstrate that Twist decoding substantially outperforms each model decoded in
isolation over various scenarios, including cases where domain-specific and
general-purpose models are both available. Twist decoding also consistently
outperforms the popular reranking heuristic where output candidates from one
model are rescored by another. We hope that our work will encourage researchers
and practitioners to examine generation models collectively, not just
independently, and to seek out models with complementary strengths to the
currently available models. Our code is available at
this https URL.

介绍了 Twist 解码这一简单通用的文字生成算法，该方法利用多样的模型来提高性能，在机器翻译和科技论文摘要等场景下表现出色，鼓励学者和实践者共同研究生成模型，寻找互补优势。