Simplicity bias is the concerning tendency of deep networks to over-depend on
simple, weakly predictive features, to the exclusion of stronger, more complex
features. This causes biased, incorrect model predictions in many real-world
applications, exacerbated by incomplete training data containing spurious
feature-label correlations. We propose a direct, interventional method for
addressing simplicity bias in DNNs, which we call the feature sieve. We aim to
automatically identify and suppress easily-computable spurious features in
lower layers of the network, thereby allowing the higher network levels to
extract and utilize richer, more meaningful representations. We provide
concrete evidence of this differential suppression & enhancement of relevant
features on both controlled datasets and real-world images, and report
substantial gains on many real-world debiasing benchmarks (11.4% relative gain
on Imagenet-A; 3.2% on BAR, etc). Crucially, we outperform many baselines that
incorporate knowledge about known spurious or biased attributes, despite our
method not using any such information. We believe that our feature sieve work
opens up exciting new research directions in automated adversarial feature
extraction & representation learning for deep networks.

本实验提出特征筛选的方法，即在神经网络的低层中自动鉴定和抑制易于计算的虚假特征，从而使高级别能够提取更丰富、更有意义的特征，这可以提高深度神经网络的识别能力和准确性，且无需依赖任何预设的知识。