Despite the remarkable abilities of Large Language Models (LLMs) to answer
questions, they often display a considerable level of overconfidence even when
the question does not have a definitive answer. To avoid providing hallucinated
answers to these unknown questions, existing studies typically investigate
approaches to refusing to answer these questions. In this work, we propose a
novel and scalable self-alignment method to utilize the LLM itself to enhance
its response-ability to different types of unknown questions, being capable of
not only refusing to answer but also providing explanation to the
unanswerability of unknown questions. Specifically, the Self-Align method first
employ a two-stage class-aware self-augmentation approach to generate a large
amount of unknown question-response data. Then we conduct disparity-driven
self-curation to select qualified data for fine-tuning the LLM itself for
aligning the responses to unknown questions as desired. Experimental results on
two datasets across four types of unknown questions validate the superiority of
the Self-Align method over existing baselines in terms of three types of task
formulation.

通过自体对齐方法，本文提出一种新颖且可扩展的方法以增强大型语言模型对不同类型未知问题的回答能力，不仅能够拒绝回答，还能提供关于无法回答的解释。实验结果验证了该方法在两个数据集上针对四种未知问题类型的三种任务形式方面优于现有基准模型。

抓住你！不要用無法回答的問題來騙我！自校准的大型語言模型用於回答未知問題

Gotcha! Don't trick me with unanswerable questions! Self-aligning Large  Language Models for Responding to Unknown Questions

Retrieve-then-read and generate-then-read are two typical solutions to handle
unknown and known questions in open-domain question-answering, while the former
retrieves necessary external knowledge and the later prompt the large language
models to generate internal known knowledge encoded in the parameters. However,
few of previous works consider the compositional unknown questions, which
consist of several known or unknown sub-questions. Thus, simple binary
classification (known or unknown) becomes sub-optimal and inefficient since it
will call external retrieval excessively for each compositional unknown
question. To this end, we propose the first Compositional unknown
Question-Answering dataset (CuQA), and introduce a Self Divide-and-Conquer
(Self-DC) framework to empower LLMs to adaptively call different methods
on-demand, resulting in better performance and efficiency. Experimental results
on two datasets (CuQA and FreshQA) demonstrate that Self-DC can achieve
comparable or even better performance with much more less retrieval times
compared with several strong baselines.

提出了第一个组合未知问题回答数据集 (CuQA) 和一种自我分割的框架 (Self-DC)，以实现适应性地调用不同方法，从而提高性能和效率。实验结果表明，Self-DC 可以与多个强基准线相比实现可比甚至更好的性能，并且检索次数更少。