Removing out-of-distribution (OOD) images from noisy images scraped from the
Internet is an important preprocessing for constructing datasets, which can be
addressed by zero-shot OOD detection with vision language foundation models
(CLIP). The existing zero-shot OOD detection setting does not consider the
realistic case where an image has both in-distribution (ID) objects and OOD
objects. However, it is important to identify such images as ID images when
collecting the images of rare classes or ethically inappropriate classes that
must not be missed. In this paper, we propose a novel problem setting called
in-distribution (ID) detection, where we identify images containing ID objects
as ID images, even if they contain OOD objects, and images lacking ID objects
as OOD images. To solve this problem, we present a new approach,
\textbf{G}lobal-\textbf{L}ocal \textbf{M}aximum \textbf{C}oncept
\textbf{M}atching (GL-MCM), based on both global and local visual-text
alignments of CLIP features, which can identify any image containing ID objects
as ID images. Extensive experiments demonstrate that GL-MCM outperforms
comparison methods on both multi-object datasets and single-object ImageNet
benchmarks.

本文提出了一种新的问题设置，称为 in-distribution（ID）检测，其中通过使用 CLIP 功能的全局和局部视觉文本对齐，我们可以将包含 ID 对象的图像识别为 ID 图像，并将缺少 ID 对象的图像识别为 OOD 图像，并且实验表明，GL-MCM 在多对象数据集和单对象 ImageNet 基准上优于比较方法。

利用视觉语言基础模型进行多对象场景下的零样本内分布检测

Zero-Shot In-Distribution Detection in Multi-Object Settings Using  Vision-Language Foundation Models

Recent state-of-the-art methods in semi-supervised learning (SSL) combine
consistency regularization with confidence-based pseudo-labeling. To obtain
high-quality pseudo-labels, a high confidence threshold is typically adopted.
However, it has been shown that softmax-based confidence scores in deep
networks can be arbitrarily high for samples far from the training data, and
thus, the pseudo-labels for even high-confidence unlabeled samples may still be
unreliable. In this work, we present a new perspective of pseudo-labeling:
instead of relying on model confidence, we instead measure whether an unlabeled
sample is likely to be "in-distribution"; i.e., close to the current training
data. To classify whether an unlabeled sample is "in-distribution" or
"out-of-distribution", we adopt the energy score from out-of-distribution
detection literature. As training progresses and more unlabeled samples become
in-distribution and contribute to training, the combined labeled and
pseudo-labeled data can better approximate the true distribution to improve the
model. Experiments demonstrate that our energy-based pseudo-labeling method,
albeit conceptually simple, significantly outperforms confidence-based methods
on imbalanced SSL benchmarks, and achieves competitive performance on
class-balanced data. For example, it produces a 4-6% absolute accuracy
improvement on CIFAR10-LT when the imbalance ratio is higher than 50. When
combined with state-of-the-art long-tailed SSL methods, further improvements
are attained.

本文提出基于能量分数的伪标记法，用于解决深度卷积神经网络中，由于 softmax 置信度无法准确反映样本离训练数据的距离，导致伪标记不可靠的问题，并在不平衡 SSL 数据集上实现了明显的精度提升。