Implicit bias describes the phenomenon where optimization-based training algorithms, without explicit regularization, show a preference for simple estimators even when more complex estimators have equal objective values. Multiple works have developed the theory of implicit bias for binary classification under the assumption that the loss satisfies an exponential tail property. However, there is a noticeable gap in analysis for multiclass classification, with only a handful of results which themselves are restricted to the cross-entropy loss. In this work, we employ the framework of Permutation Equivariant and Relative Margin-based (PERM) losses [Wang and Scott, 2024] to introduce a multiclass extension of the exponential tail property. This class of losses includes not only cross-entropy but also other losses. Using this framework, we extend the implicit bias result of Soudry et al. [2018] to multiclass classification. Furthermore, our proof techniques closely mirror those of the binary case, thus illustrating the power of the PERM framework for bridging the binary-multiclass gap.

本研究解决了多类分类中隐性偏差理论的不足，特别是在现有研究主要集中于二类分类的情况下。作者通过引入排列等变性和相对边际（PERM）损失的框架，扩展了指数尾性特性到多类分类，并将相关结果应用于此领域。研究结果显示，PERM框架有效地填补了二类与多类分类之间的理论空白。 

梯度下降在可分多类数据上的隐性偏差