Current Large Language Models (LLMs) face inherent limitations due to their
pre-defined context lengths, which impede their capacity for multi-hop
reasoning within extensive textual contexts. While existing techniques like
Retrieval-Augmented Generation (RAG) have attempted to bridge this gap by
sourcing external information, they fall short when direct answers are not
readily available. We introduce a novel approach that re-imagines information
retrieval through dynamic in-context editing, inspired by recent breakthroughs
in knowledge editing. By treating lengthy contexts as malleable external
knowledge, our method interactively gathers and integrates relevant
information, thereby enabling LLMs to perform sophisticated reasoning steps.
Experimental results demonstrate that our method effectively empowers
context-limited LLMs, such as Llama2, to engage in multi-hop reasoning with
improved performance, which outperforms state-of-the-art context window
extrapolation methods and even compares favorably to more advanced commercial
long-context models. Our interactive method not only enhances reasoning
capabilities but also mitigates the associated training and computational
costs, making it a pragmatic solution for enhancing LLMs' reasoning within
expansive contexts.

通过动态上下文编辑的方式，我们引入了一种新的方法，将信息检索重新设想，使长文本上下文成为可塑的外部知识，并通过与最新的知识编辑技术相结合，与有关信息进行交互式收集和整合，从而使大型语言模型能够进行复杂的推理步骤，有效增强了推理能力。