Language models generate responses by producing a series of tokens in
immediate succession: the $(K+1)^{th}$ token is an outcome of manipulating $K$
hidden vectors per layer, one vector per preceding token. What if instead we
were to let the model manipulate say, $K+10$ hidden vectors, before it outputs
the $(K+1)^{th}$ token? We operationalize this idea by performing training and
inference on language models with a (learnable) $\textit{pause}$ token, a
sequence of which is appended to the input prefix. We then delay extracting the
model's outputs until the last pause token is seen, thereby allowing the model
to process extra computation before committing to an answer. We empirically
evaluate $\textit{pause-training}$ on decoder-only models of 1B and 130M
parameters with causal pretraining on C4, and on downstream tasks covering
reasoning, question-answering, general understanding and fact recall. Our main
finding is that inference-time delays show gains when the model is both
pre-trained and finetuned with delays. For the 1B model, we witness gains on 8
of 9 tasks, most prominently, a gain of $18\%$ EM score on the QA task of
SQuAD, $8\%$ on CommonSenseQA and $1\%$ accuracy on the reasoning task of
GSM8k. Our work raises a range of conceptual and practical future research
questions on making delayed next-token prediction a widely applicable new
paradigm.

使用一个可学习的 “暂停” 标记和推迟提取模型输出的方法，我们在语言模型上执行训练和推理，并观察到模型在预训练和微调过程中实现了推理时间延迟的增益，这对各种推理、问答和基本理解任务都有积极的影响，进一步将延迟下一个标记预测的方法应用于各种应用场景可能引发更多概念性和实践性的未来研究问题。