In an information-seeking conversation, a user converses with an agent to ask
a series of questions that can often be under- or over-specified. An ideal
agent would first identify that they were in such a situation by searching
through their underlying knowledge source and then appropriately interacting
with a user to resolve it. However, most existing studies either fail to or
artificially incorporate such agent-side initiatives. In this work, we present
INSCIT (pronounced Insight), a dataset for information-seeking conversations
with mixed-initiative interactions. It contains a total of 4.7K user-agent
turns from 805 human-human conversations where the agent searches over
Wikipedia and either asks for clarification or provides relevant information to
address user queries. We define two subtasks, namely evidence passage
identification and response generation, as well as a new human evaluation
protocol to assess the model performance. We report results of two strong
baselines based on state-of-the-art models of conversational knowledge
identification and open-domain question answering. Both models significantly
underperform humans and fail to generate coherent and informative responses,
suggesting ample room for improvement in future studies.

该论文介绍了 INSCIT 数据集，该数据集用于信息搜索对话的研究，其中包含 805 个人 - 人之间的对话中的 4.7K 用户 - 代理器的交互，定义了证据段落识别和响应生成的两个子任务，以及一个新的人类评估协议以评估模型的性能，证实现有大量改进的空间。