Large language models (LLMs) have shown impressive results across a variety of tasks while requiring little or no direct supervision. Further, there is mounting evidence that LLMs may have potential in information-seeking scenarios. We believe the ability of an LLM to attribute the text that it generates is likely to be crucial for both system developers and users in this setting. We propose and study Attributed QA as a key first step in the development of attributed LLMs. We develop a reproducable evaluation framework for the task, using human annotations as a gold standard and a correlated automatic metric that we show is suitable for development settings. We describe and benchmark a broad set of architectures for the task. Our contributions give some concrete answers to two key questions (How to measure attribution?, and How well do current state-of-the-art methods perform on attribution?), and give some hints as to how to address a third key question (How to build LLMs with attribution?).

本文研究基于属性的LLM发展，在开发Attributed LLMs的第一步骤中提出可重复的Attributed QA评估框架并评估多种结构。实验结果探讨了如何度量归属（attribution）以及现有方法在归属方面的表现如何，并提出了建立带归属特性的LLMs的可能方向。

属性问答：针对属性化大语言模型的评估和建模