Most existing out-of-distribution (OOD) detection benchmarks classify samples
with novel labels as the OOD data. However, some marginal OOD samples actually
have close semantic contents to the in-distribution (ID) sample, which makes
determining the OOD sample a Sorites Paradox. In this paper, we construct a
benchmark named Incremental Shift OOD (IS-OOD) to address the issue, in which
we divide the test samples into subsets with different semantic and covariate
shift degrees relative to the ID dataset. The data division is achieved through
a shift measuring method based on our proposed Language Aligned Image feature
Decomposition (LAID). Moreover, we construct a Synthetic Incremental Shift
(Syn-IS) dataset that contains high-quality generated images with more diverse
covariate contents to complement the IS-OOD benchmark. We evaluate current OOD
detection methods on our benchmark and find several important insights: (1) The
performance of most OOD detection methods significantly improves as the
semantic shift increases; (2) Some methods like GradNorm may have different OOD
detection mechanisms as they rely less on semantic shifts to make decisions;
(3) Excessive covariate shifts in the image are also likely to be considered as
OOD for some methods. Our code and data are released in
this https URL

本研究构建了一个名为 Incremental Shift OOD（IS-OOD）的基准，通过基于提出的语言对齐图像特征分解（LAID）的移位测量方法，将测试样本分成不同的子集，相对于 ID 数据集具有不同的语义和协变量移位程度，以解决当前存在的超出分布（OOD）检测基准中的 Sorites 悖论问题。此外，还构建了包含更多多样的协变量内容的高质量生成图像的 Synthetic Incremental Shift（Syn-IS）数据集，以补充 IS-OOD 基准。研究评估了在本基准上进行的当前 OOD 检测方法，并发现几个重要结论：（1）大多数 OOD 检测方法的性能随着语义移位的增加而显著提高；（2）某些方法如 GradNorm 在做出决策时较少依赖于语义移位，可能具有不同的 OOD 检测机制；（3）图像中过度的协变量移位也可能被某些方法视为 OOD。