We study the asymptotic generalization of an overparameterized linear model
for multiclass classification under the Gaussian covariates bi-level model
introduced in Subramanian et al.~'22, where the number of data points,
features, and classes all grow together. We fully resolve the conjecture posed
in Subramanian et al.~'22, matching the predicted regimes for generalization.
Furthermore, our new lower bounds are akin to an information-theoretic strong
converse: they establish that the misclassification rate goes to 0 or 1
asymptotically. One surprising consequence of our tight results is that the
min-norm interpolating classifier can be asymptotically suboptimal relative to
noninterpolating classifiers in the regime where the min-norm interpolating
regressor is known to be optimal.
The key to our tight analysis is a new variant of the Hanson-Wright
inequality which is broadly useful for multiclass problems with sparse labels.
As an application, we show that the same type of analysis can be used to
analyze the related multilabel classification problem under the same bi-level
ensemble.

本文研究了高超参数线性模型在多类别高斯协变量下的渐近泛化，包括对 Subramanian 等人所提出的双层模型的研究，提出了新的下界，证明了该模型的渐近一致性，并提供了一个在稀疏标签多类问题中广泛适用的 Hanson-Wright 不等式的变体。

过参数化线性模型的多类分类精确渐近推广

Precise Asymptotic Generalization for Multiclass Classification with  Overparameterized Linear Models

To better understand catastrophic forgetting, we study fitting an
overparameterized linear model to a sequence of tasks with different input
distributions. We analyze how much the model forgets the true labels of earlier
tasks after training on subsequent tasks, obtaining exact expressions and
bounds. We establish connections between continual learning in the linear
setting and two other research areas: alternating projections and the Kaczmarz
method. In specific settings, we highlight differences between forgetting and
convergence to the offline solution as studied in those areas. In particular,
when T tasks in d dimensions are presented cyclically for k iterations, we
prove an upper bound of T^2 * min{1/sqrt(k), d/k} on the forgetting. This
stands in contrast to the convergence to the offline solution, which can be
arbitrarily slow according to existing alternating projection results. We
further show that the T^2 factor can be lifted when tasks are presented in a
random ordering.

研究过拟合线性模型在不同输入分布下对一系列任务进行训练时出现的灾难性遗忘现象，探究其与不断学习、交错投影和 Kaczmarz 方法之间的联系，并针对具体情况提出精确的遗忘量度和上限。

线性回归中灾难性遗忘能有多严重？

How catastrophic can catastrophic forgetting be in linear regression?

We compare classification and regression tasks in an overparameterized linear
model with Gaussian features. On the one hand, we show that with sufficient
overparameterization all training points are support vectors: solutions
obtained by least-squares minimum-norm interpolation, typically used for
regression, are identical to those produced by the hard-margin support vector
machine (SVM) that minimizes the hinge loss, typically used for training
classifiers. On the other hand, we show that there exist regimes where these
interpolating solutions generalize well when evaluated by the 0-1 test loss
function, but do not generalize if evaluated by the square loss function, i.e.
they approach the null risk. Our results demonstrate the very different roles
and properties of loss functions used at the training phase (optimization) and
the testing phase (generalization).

这篇研究论文探讨了使用高斯特征的超参数化线性模型在分类和回归任务中的表现，发现当过度参数化时，所有训练点都成为支持向量，从而演化出一个法向量的模型，比起普通的最小二乘法和解决分类问题的 SVM，更适用于两者的情况，并讨论了在不同 loss function 下，训练（优化）和测试（归纳）階段使用的方法的不同作用和特性。