Despite the successes of large language models (LLMs), they exhibit significant drawbacks, particularly when processing long contexts. Their inference cost scales quadratically with respect to sequence length, making it expensive for deployment in some real-world text processing applications, such as retrieval-augmented generation (RAG). Additionally, LLMs also exhibit the "distraction phenomenon," where irrelevant context in the prompt degrades output quality. To address these drawbacks, we propose a novel RAG prompting methodology, superposition prompting, which can be directly applied to pre-trained transformer-based LLMs without the need for fine-tuning. At a high level, superposition prompting allows the LLM to process input documents in parallel prompt paths, discarding paths once they are deemed irrelevant. We demonstrate the capability of our method to simultaneously enhance time efficiency across a variety of question-answering benchmarks using multiple pre-trained LLMs. Furthermore, our technique significantly improves accuracy when the retrieved context is large relative the context the model was trained on. For example, our approach facilitates an 93x reduction in compute time while improving accuracy by 43\% on the NaturalQuestions-Open dataset with the MPT-7B instruction-tuned model over naive RAG.

我们提出了一种新颖的RAG提示方法，即超级叠加提示，可以直接应用于预训练的基于transformer的大语言模型，无需进行精调，以解决大语言模型在处理长上下文时的推理成本呈二次比例增长、输出质量受无关上下文干扰的问题。我们的方法可以同时提高各种问答基准测试的时间效率，并且在检索到的上下文相对于模型训练时的上下文更大的情况下显著提高准确性。

叠加提示：改善和加速检索增强生成