Existing methods for creating source-grounded information-seeking dialog
datasets are often costly and hard to implement due to their sole reliance on
human annotators. We propose combining large language models (LLMs) prompting
with human expertise for more efficient and reliable data generation. Instead
of the labor-intensive Wizard-of-Oz (WOZ) method, where two annotators generate
a dialog from scratch, role-playing agent and user, we use LLM generation to
simulate the two roles. Annotators then verify the output and augment it with
attribution data. We demonstrate our method by constructing MISeD -- Meeting
Information Seeking Dialogs dataset -- the first information-seeking dialog
dataset focused on meeting transcripts. Models finetuned with MISeD demonstrate
superior performance on our test set, as well as on a novel fully-manual WOZ
test set and an existing query-based summarization benchmark, suggesting the
utility of our approach.

结合大型语言模型与人类专业知识，通过将角色扮演代理和用户的对话模拟为大型语言模型生成，然后由人类专家验证并补充归属数据，以更高效可靠地生成数据，并构建了集中在会议记录的首个信息检索对话数据集 MISeD，其中模型在测试集、手动 WOZ 测试集和现有基于查询的摘要基准上的优越性能表明我们方法的效用。