Out-of-distribution (OOD) problems in few-shot classification (FSC) occur
when novel classes sampled from testing distributions differ from base classes
drawn from training distributions, which considerably degrades the performance
of deep learning models deployed in real-world applications. Recent studies
suggest that the OOD problems in FSC mainly including: (a) cross-domain
few-shot classification (CD-FSC) and (b) spurious-correlation few-shot
classification (SC-FSC). Specifically, CD-FSC occurs when a classifier learns
transferring knowledge from base classes drawn from seen training distributions
but recognizes novel classes sampled from unseen testing distributions. In
contrast, SC-FSC arises when a classifier relies on non-causal features (or
contexts) that happen to be correlated with the labels (or concepts) in base
classes but such relationships no longer hold during the model deployment.
Despite CD-FSC has been extensively studied, SC-FSC remains understudied due to
lack of the corresponding evaluation benchmarks. To this end, we present Meta
Concept Context (MetaCoCo), a benchmark with spurious-correlation shifts
collected from real-world scenarios. Moreover, to quantify the extent of
spurious-correlation shifts of the presented MetaCoCo, we further propose a
metric by using CLIP as a pre-trained vision-language model. Extensive
experiments on the proposed benchmark are performed to evaluate the
state-of-the-art methods in FSC, cross-domain shifts, and self-supervised
learning. The experimental results show that the performance of the existing
methods degrades significantly in the presence of spurious-correlation shifts.
We open-source all codes of our benchmark and hope that the proposed MetaCoCo
can facilitate future research on spurious-correlation shifts problems in FSC.
The code is available at: this https URL

少样本分类中的异构问题（OOD）会导致模型性能下降，而交叉领域少样本分类（CD-FSC）和虚假相关少样本分类（SC-FSC）是 OOD 问题的主要组成部分。为了解决 SC-FSC 问题，我们提出了一个真实场景下收集的含有虚假相关偏移的基准测试 MetaCoCo，并使用 CLIP 作为预训练视觉语言模型来量化 MetaCoCo 的虚假相关偏移程度。实验结果表明，现有方法在存在虚假相关偏移的情况下性能显著下降。我们开源了基准测试的所有代码，希望 MetaCoCo 能促进未来在 SC-FSC 问题上的研究。