We revisit the likelihood ratio between a pretrained large language model
(LLM) and its finetuned variant as a criterion for out-of-distribution (OOD)
detection. The intuition behind such a criterion is that, the pretrained LLM
has the prior knowledge about OOD data due to its large amount of training
data, and once finetuned with the in-distribution data, the LLM has sufficient
knowledge to distinguish their difference. Leveraging the power of LLMs, we
show that, for the first time, the likelihood ratio can serve as an effective
OOD detector. Moreover, we apply the proposed LLM-based likelihood ratio to
detect OOD questions in question-answering (QA) systems, which can be used to
improve the performance of specialized LLMs for general questions. Given that
likelihood can be easily obtained by the loss functions within contemporary
neural network frameworks, it is straightforward to implement this approach in
practice. Since both the pretrained LLMs and its various finetuned models are
available, our proposed criterion can be effortlessly incorporated for OOD
detection without the need for further training. We conduct comprehensive
evaluation across on multiple settings, including far OOD, near OOD, spam
detection, and QA scenarios, to demonstrate the effectiveness of the method.

通过重新审视预训练大型语言模型和其微调变体之间的似然比作为一种区分所需分布检测的标准，我们展示了似然比可以作为一种有效的 OOD 检测器，并将其应用于问题回答系统中以改善 LLMs 在一般问题上的性能。