The increasing demand for the deployment of LLMs in information-seeking
scenarios has spurred efforts in creating verifiable systems, which generate
responses to queries along with supporting evidence. In this paper, we explore
the attribution capabilities of plan-based models which have been recently
shown to improve the faithfulness, grounding, and controllability of generated
text. We conceptualize plans as a sequence of questions which serve as
blueprints of the generated content and its organization. We propose two
attribution models that utilize different variants of blueprints, an
abstractive model where questions are generated from scratch, and an extractive
model where questions are copied from the input. Experiments on long-form
question-answering show that planning consistently improves attribution
quality. Moreover, the citations generated by blueprint models are more
accurate compared to those obtained from LLM-based pipelines lacking a planning
component.

我们在这篇论文中探讨了最近显示对于生成的文本的逼真性、基础性和可控性有所改进的计划型模型的归因能力，该模型将计划概念化为一系列问题，旨在作为生成内容及其组织的蓝图。我们提出了两个利用不同蓝图变体的归因模型，一个是从头开始生成问题的抽象模型，另一个是从输入中复制问题的抽取模型。在长文问题回答的实验中，规划始终能够提高归因质量，此外，蓝图模型生成的引用文献与缺乏规划组件的基于 LLM 的流程获得的引用相比更加准确。

学习计划与引文生成文本

Learning to Plan and Generate Text with Citations

The rise of large language models (LLMs) had a transformative impact on
search, ushering in a new era of search engines that are capable of generating
search results in natural language text, imbued with citations for supporting
sources. Building generative information-seeking models demands openly
accessible datasets, which currently remain lacking. In this paper, we
introduce a new dataset, HAGRID (Human-in-the-loop Attributable Generative
Retrieval for Information-seeking Dataset) for building end-to-end generative
information-seeking models that are capable of retrieving candidate quotes and
generating attributed explanations. Unlike recent efforts that focus on human
evaluation of black-box proprietary search engines, we built our dataset atop
the English subset of MIRACL, a publicly available information retrieval
dataset. HAGRID is constructed based on human and LLM collaboration. We first
automatically collect attributed explanations that follow an in-context
citation style using an LLM, i.e. GPT-3.5. Next, we ask human annotators to
evaluate the LLM explanations based on two criteria: informativeness and
attributability. HAGRID serves as a catalyst for the development of
information-seeking models with better attribution capabilities.

通过构建 HAGRID 数据集，该文研究了大型语言模型对搜索引擎的影响，为开发具有更好归因能力的信息寻找模型提供了新的数据资源。