Large language models (LLMs) have a surprising failure: when trained on "A
has a feature B", they do not generalize to "B is a feature of A", which is
termed the Reversal Curse. Even when training with trillions of tokens this
issue still appears due to Zipf's law - hence even if we train on the entire
internet. This work proposes an alternative training scheme, called reverse
training, whereby all words are used twice, doubling the amount of available
tokens. The LLM is trained in both forward and reverse directions by reversing
the training strings while preserving (i.e., not reversing) chosen substrings,
such as entities. We show that data-matched reverse-trained models provide
superior performance to standard models on standard tasks, and compute-matched
reverse-trained models provide far superior performance on reversal tasks,
helping resolve the reversal curse issue.

大型语言模型（LLMs）在 “A 具有特征 B” 的训练中无法推广到 “B 是 A 的特征”，这称为反转诅咒。本研究提出了一种替代训练方案，称为反向训练，通过将训练字符串进行反转并保留（即不反转）选择的子字符串（如实体）来同时以正向和反向方式训练 LLM，以加倍可用令牌的数量。我们证明，数据匹配的反向训练模型在标准任务上提供了优越的性能，而计算匹配的反向训练模型在反转任务上提供了远超过标准模型的性能，有助于解决反转诅咒问题。