Deep neural networks are increasingly used in a wide range of technologies
and services, but remain highly susceptible to out-of-distribution (OOD)
samples, that is, drawn from a different distribution than the original
training set. A common approach to address this issue is to endow deep neural
networks with the ability to detect OOD samples. Several benchmarks have been
proposed to design and validate OOD detection techniques. However, many of them
are based on far-OOD samples drawn from very different distributions, and thus
lack the complexity needed to capture the nuances of real-world scenarios. In
this work, we introduce a comprehensive benchmark for OOD detection, based on
ImageNet and Places365, that assigns individual classes as in-distribution or
out-of-distribution depending on the semantic similarity with the training set.
Several techniques can be used to determine which classes should be considered
in-distribution, yielding benchmarks with varying properties. Experimental
results on different OOD detection techniques show how their measured efficacy
depends on the selected benchmark and how confidence-based techniques may
outperform classifier-based ones on near-OOD samples.

深度神经网络在各种技术和服务中得到越来越广泛的应用，但其容易受到来自训练集不同分布的样本的干扰，而常见的解决方法是使深度神经网络具备检测这种样本的能力。本文提出了一种基于 ImageNet 和 Places365 的全面评估标准，根据与训练集的语义相似性，将个别类别分为内部分布和外部分布，通过不同的技术确定哪些类别应被视为内部分布，得到具备不同性质的评估标准。不同的 ODD 检测技术在不同的评估标准下的实验结果表明，它们的有效性取决于所选择的评估标准，而基于置信度的技术在接近 ODD 样本上可能优于基于分类器的技术。