Common explanations for shortcut learning assume that the shortcut improves
prediction under the training distribution but not in the test distribution.
Thus, models trained via the typical gradient-based optimization of
cross-entropy, which we call default-ERM, utilize the shortcut. However, even
when the stable feature determines the label in the training distribution and
the shortcut does not provide any additional information, like in perception
tasks, default-ERM still exhibits shortcut learning. Why are such solutions
preferred when the loss for default-ERM can be driven to zero using the stable
feature alone? By studying a linear perception task, we show that default-ERM's
preference for maximizing the margin leads to models that depend more on the
shortcut than the stable feature, even without overparameterization. This
insight suggests that default-ERM's implicit inductive bias towards max-margin
is unsuitable for perception tasks. Instead, we develop an inductive bias
toward uniform margins and show that this bias guarantees dependence only on
the perfect stable feature in the linear perception task. We develop loss
functions that encourage uniform-margin solutions, called margin control
(MARG-CTRL). MARG-CTRL mitigates shortcut learning on a variety of vision and
language tasks, showing that better inductive biases can remove the need for
expensive two-stage shortcut-mitigating methods in perception tasks.

对于感知任务而言，默认的 ERM 的隐含归纳偏差偏向于最大化边界学习模型，该偏差通过鼓励均匀边界解决方案可以消除 shortcut 学习，并在视觉和语言任务中证明更好的归纳偏差可以消除费时的两阶段 shortcut 缓解方法。