We introduce thoughts of words (ToW), a novel training-time data-augmentation method for next-word prediction. ToW views next-word prediction as a core reasoning task and injects fine-grained thoughts explaining what the next word should be and how it is related to the previous contexts in pre-training texts. Our formulation addresses two fundamental drawbacks of existing next-word prediction learning schemes: they induce factual hallucination and are inefficient for models to learn the implicit reasoning processes in raw texts. While there are many ways to acquire such thoughts of words, we explore the first step of acquiring ToW annotations through distilling from larger models. After continual pre-training with only 70K ToW annotations, we effectively improve models' reasoning performances by 7% to 9% on average and reduce model hallucination by up to 10%. At the same time, ToW is entirely agnostic to tasks and applications, introducing no additional biases on labels or semantics.

本文提出了一种名为“词的思考”（ToW）的新型数据增强方法，旨在解决现有下一词预测学习方案的事实幻觉和效率低下问题。通过从大型模型中提取ToW注释，在仅使用70K ToW注释的情况下，模型的推理能力提高了7%至9%，同时减少了高达10%的幻觉现象，展示了显著的潜在影响。

词的思考：提升大型语言模型推理能力