Ambiguous questions are a challenge for Question Answering models, as they
require answers that cover multiple interpretations of the original query. To
this end, these models are required to generate long-form answers that often
combine conflicting pieces of information. Although recent advances in the
field have shown strong capabilities in generating fluent responses, certain
research questions remain unanswered. Does model/data scaling improve the
answers' quality? Do automated metrics align with human judgment? To what
extent do these models ground their answers in evidence? In this study, we aim
to thoroughly investigate these aspects, and provide valuable insights into the
limitations of the current approaches. To aid in reproducibility and further
extension of our work, we open-source our code at
this https URL

研究 Question Answering 模型回答模糊问题的挑战，探讨模型 / 数据扩展和自动评估指标对模型质量的影响，并研究模型答案的证据根据，旨在提供有关当前方法局限性的宝贵见解。

模型分析与评估：歧义问题回答

Model Analysis & Evaluation for Ambiguous Question Answering

Long-form answers, consisting of multiple sentences, can provide nuanced and
comprehensive answers to a broader set of questions. To better understand this
complex and understudied task, we study the functional structure of long-form
answers collected from three datasets, ELI5, WebGPT and Natural Questions. Our
main goal is to understand how humans organize information to craft complex
answers. We develop an ontology of six sentence-level functional roles for
long-form answers, and annotate 3.9k sentences in 640 answer paragraphs.
Different answer collection methods manifest in different discourse structures.
We further analyze model-generated answers -- finding that annotators agree
less with each other when annotating model-generated answers compared to
annotating human-written answers. Our annotated data enables training a strong
classifier that can be used for automatic analysis. We hope our work can
inspire future research on discourse-level modeling and evaluation of long-form
QA systems.

本篇论文通过研究三个数据集中的长篇答案，开发了一个六个句子级功能角色的本体论，并进一步分析了模型生成的答案。通过对 3.9k 句子和 640 个答案段落进行注释，我们希望能够鼓励未来的学者进行话语层模型的研究和长篇问答系统的评估。