Deep neural networks can be unreliable in the real world especially when they heavily use spurious features for their predictions. Recently, Singla & Feizi (2022) introduced the Salient Imagenet dataset by annotating and localizing core and spurious features of ~52k samples from 232 classes of Imagenet. While this dataset is useful for evaluating the reliance of pretrained models on spurious features, its small size limits its usefulness for training models. In this work, we first introduce the Salient Imagenet-1M dataset with more than 1 million soft masks localizing core and spurious features for all 1000 Imagenet classes. Using this dataset, we first evaluate the reliance of several Imagenet pretrained models (42 total) on spurious features and observe that: (i) transformers are more sensitive to spurious features compared to Convnets, (ii) zero-shot CLIP transformers are highly susceptible to spurious features. Next, we introduce a new learning paradigm called Core Risk Minimization (CoRM) whose objective ensures that the model predicts a class using its core features. We evaluate different computational approaches for solving CoRM and achieve significantly higher (+12%) core accuracy (accuracy when non-core regions corrupted using noise) with no drop in clean accuracy compared to models trained via Empirical Risk Minimization.

本文中，我们介绍了 Salient Imagenet-1M 数据集，并评估了几个预训练模型对于虚假特征的依赖性；而后，我们提出了一种新的学习范式叫做核风险最小化，其目的是确保模型使用核心特征对类进行预测。我们评估了不同的计算方法，发现训练 Core Risk Minimization 模型的核心准确率比经验风险最小化模型的准确率高约12％，并且干扰后没有降低干净准确率。

基于显著性 ImageNet 的核心风险最小化