Large language models have been shown to struggle with limited context memory
and multi-step reasoning. We propose a simple method for solving both of these
problems by allowing the model to take Self-Notes. Unlike recent scratchpad
approaches, the model can deviate from the input context at any time to
explicitly think. This allows the model to recall information and perform
reasoning on the fly as it reads the context, thus extending its memory and
enabling multi-step reasoning. Our experiments on multiple tasks demonstrate
that our method can successfully generalize to longer and more complicated
instances from their training setup by taking Self-Notes at inference time.

通过让模型进行自我记录 (Self-Notes) 实现大型语言模型更好的处理上下文记忆和多步推理问题，能够在阅读上下文的同时，记忆信息和执行推理，从而拓展其记忆能力和实现多步推理，并在多个任务中展示了良好的推广能力。

利用自身笔记学习推理和记忆

Learning to Reason and Memorize with Self-Notes

Dialogue contexts are proven helpful in the spoken language understanding
(SLU) system and they are typically encoded with explicit memory
representations. However, most of the previous models learn the context memory
with only one objective to maximizing the SLU performance, leaving the context
memory under-exploited. In this paper, we propose a new dialogue logistic
inference (DLI) task to consolidate the context memory jointly with SLU in the
multi-task framework. DLI is defined as sorting a shuffled dialogue session
into its original logical order and shares the same memory encoder and
retrieval mechanism as the SLU model. Our experimental results show that
various popular contextual SLU models can benefit from our approach, and
improvements are quite impressive, especially in slot filling.

本研究提出了一种新的对话逻辑推理（DLI）任务，在多任务框架中共同巩固上下文记忆和 SLU 模型，实验结果表明各种领先的上下文 SLU 模型可以从我们的方法中获益，尤其是在槽填充方面。