This document presents a detailed description of the challenge on clarifying
questions for dialogue systems (ClariQ). The challenge is organized as part of
the Conversational AI challenge series (ConvAI3) at Search Oriented
Conversational AI (SCAI) EMNLP workshop in 2020. The main aim of the
conversational systems is to return an appropriate answer in response to the
user requests. However, some user requests might be ambiguous. In IR settings
such a situation is handled mainly thought the diversification of the search
result page. It is however much more challenging in dialogue settings with
limited bandwidth. Therefore, in this challenge, we provide a common evaluation
framework to evaluate mixed-initiative conversations. Participants are asked to
rank clarifying questions in an information-seeking conversations. The
challenge is organized in two stages where in Stage 1 we evaluate the
submissions in an offline setting and single-turn conversations. Top
participants of Stage 1 get the chance to have their model tested by human
annotators.

本文详细介绍了对话系统提出澄清问题的挑战（ClariQ），为了解决信息查找中遇到的模糊问题，提供通用的评估框架以评估混合倡议式对话。

ConvAI3: 为开放域对话系统生成澄清问题（ClariQ）

ConvAI3: Generating Clarifying Questions for Open-Domain Dialogue  Systems (ClariQ)

Conversational assistants are being progressively adopted by the general
population. However, they are not capable of handling complicated
information-seeking tasks that involve multiple turns of information exchange.
Due to the limited communication bandwidth in conversational search, it is
important for conversational assistants to accurately detect and predict user
intent in information-seeking conversations. In this paper, we investigate two
aspects of user intent prediction in an information-seeking setting. First, we
extract features based on the content, structural, and sentiment
characteristics of a given utterance, and use classic machine learning methods
to perform user intent prediction. We then conduct an in-depth feature
importance analysis to identify key features in this prediction task. We find
that structural features contribute most to the prediction performance. Given
this finding, we construct neural classifiers to incorporate context
information and achieve better performance without feature engineering. Our
findings can provide insights into the important factors and effective methods
of user intent prediction in information-seeking conversations.

研究了信息检索中用户意图识别的两个方面，提取基于内容，结构和情感特征的特征，利用经典机器学习方法执行用户意图预测，使用特征重要性分析确定了结构特征对预测的贡献最大，并构建了神经分类器以提高性能。

信息检索对话中的用户意图预测

User Intent Prediction in Information-seeking Conversations

Understanding and characterizing how people interact in information-seeking
conversations is crucial in developing conversational search systems. In this
paper, we introduce a new dataset designed for this purpose and use it to
analyze information-seeking conversations by user intent distribution,
co-occurrence, and flow patterns. The MSDialog dataset is a labeled dialog
dataset of question answering (QA) interactions between information seekers and
providers from an online forum on Microsoft products. The dataset contains more
than 2,000 multi-turn QA dialogs with 10,000 utterances that are annotated with
user intent on the utterance level. Annotations were done using crowdsourcing.
With MSDialog, we find some highly recurring patterns in user intent during an
information-seeking process. They could be useful for designing conversational
search systems. We will make our dataset freely available to encourage
exploration of information-seeking conversation models.

本研究介绍了一个新的数据集，用于分析信息搜寻对话的用户意图分布、共同出现和流程模式，并利用 MSDialog 数据集发现了一些可用于设计对话式搜索系统的高频用户意图模式。