Though state-of-the-art (SOTA) NLP systems have achieved remarkable
performance on a variety of language understanding tasks, they primarily focus
on questions that have a correct and a definitive answer. However, in
real-world applications, users often ask questions that don't have a definitive
answer. Incorrectly answering such questions certainly hampers a system's
reliability and trustworthiness. Can SOTA models accurately identify such
questions and provide a reasonable response?
To investigate the above question, we introduce QnotA, a dataset consisting
of five different categories of questions that don't have definitive answers.
Furthermore, for each QnotA instance, we also provide a corresponding QA
instance i.e. an alternate question that ''can be'' answered. With this data,
we formulate three evaluation tasks that test a system's ability to 'identify',
'distinguish', and 'justify' QnotA questions. Through comprehensive
experiments, we show that even SOTA models including GPT-3 and Flan T5 do not
fare well on these tasks and lack considerably behind the human performance
baseline. We conduct a thorough analysis which further leads to several
interesting findings. Overall, we believe our work and findings will encourage
and facilitate further research in this important area and help develop more
robust models.

尽管最先进的自然语言处理（NLP）系统在各种语言理解任务上取得了显著的表现，但它们主要关注那些有正确和明确答案的问题。然而，在现实世界的应用中，用户经常会问一些没有明确答案的问题。最新研究表明，即使是 SOTA 模型（包括 GPT-3 和 Flan T5）在正确识别和回答这类问题上也表现不佳，远远落后于人类基准水平。本研究介绍了一个包含五类没有明确答案的问题的数据集 QnotA，同时提供了每个 QnotA 实例的一个可回答问题。我们通过综合实验表明，我们的工作和发现将鼓励和促进进一步在这一重要领域的研究，并有助于开发更健壮的模型。