Deep-learning models can extract a rich assortment of features from data.
Which features a model uses depends not only on predictivity-how reliably a
feature indicates train-set labels-but also on availability-how easily the
feature can be extracted, or leveraged, from inputs. The literature on shortcut
learning has noted examples in which models privilege one feature over another,
for example texture over shape and image backgrounds over foreground objects.
Here, we test hypotheses about which input properties are more available to a
model, and systematically study how predictivity and availability interact to
shape models' feature use. We construct a minimal, explicit generative
framework for synthesizing classification datasets with two latent features
that vary in predictivity and in factors we hypothesize to relate to
availability, and quantify a model's shortcut bias-its over-reliance on the
shortcut (more available, less predictive) feature at the expense of the core
(less available, more predictive) feature. We find that linear models are
relatively unbiased, but introducing a single hidden layer with ReLU or Tanh
units yields a bias. Our empirical findings are consistent with a theoretical
account based on Neural Tangent Kernels. Finally, we study how models used in
practice trade off predictivity and availability in naturalistic datasets,
discovering availability manipulations which increase models' degree of
shortcut bias. Taken together, these findings suggest that the propensity to
learn shortcut features is a fundamental characteristic of deep nonlinear
architectures warranting systematic study given its role in shaping how models
solve tasks.

深度学习模型从数据中提取多种多样的特征，但这些特征的使用不仅取决于其预测能力（即特征如何可靠地指示训练数据标签），还取决于其可提取性（即特征从输入中容易提取或利用的程度）。研究表明，某些模型更喜欢使用某些特定的特征，例如纹理优先于形状，图像背景优先于前景对象。本研究通过构建一个具有预测性和可提取性差异的最小显性生成框架，并量化了模型的 Shortcut Bias（即过度依赖快捷特征而忽视核心特征的程度），从而系统研究了预测性和可提取性如何相互作用来影响模型的特征使用。实证结果表明，线性模型相对没有偏见，但引入具有 ReLU 或 Tanh 单隐藏层的模型则会产生偏见。最后，本研究还研究了在实际使用中，模型是如何在自然数据集中权衡预测性和可提取性，并发现了增加模型 Shortcut Bias 程度的可提取性操作。综上所述，这些发现表明，在深度非线性架构中学习快捷特征的倾向是一个基本特征，值得进行系统研究，因为它影响模型解决任务的方式。