Neural Collapse is a recently observed geometric structure that emerges in
the final layer of neural network classifiers. Specifically, Neural Collapse
states that at the terminal phase of neural networks training, 1) the
intra-class variability of last-layer features tends to zero, 2) the class
feature means form an Equiangular Tight Frame (ETF), 3) last-layer class
features and weights becomes equal up the scaling, and 4) classification
behavior collapses to the nearest class center (NCC) decision rule. This paper
investigates the effect of batch normalization and weight decay on the
emergence of Neural Collapse. We propose the geometrically intuitive
intra-class and inter-class cosine similarity measure which captures multiple
core aspects of Neural Collapse. With this measure, we provide theoretical
guarantees of Neural Collapse emergence with last-layer batch normalization and
weight decay when the regularized cross-entropy loss is near optimal. We also
perform further experiments to show that the Neural Collapse is most
significant in models with batch normalization and high weight-decay values.
Collectively, our results imply that batch normalization and weight decay may
be fundamental factors in the emergence of Neural Collapse.

神经崩溃是最后一层神经网络分类器中出现的几何结构，本文研究了批归一化和权重衰减对神经崩溃的影响，并提出了几何直观的类内和类间余弦相似度度量来捕捉神经崩溃的核心方面。结果表明，批归一化和权重衰减可能是神经崩溃出现的基本因素。

探索神经衰竭：批标准化和权重衰减的影响

Towards Understanding Neural Collapse: The Effects of Batch  Normalization and Weight Decay

Each year, thousands of people learn new visual categorization tasks --
radiologists learn to recognize tumors, birdwatchers learn to distinguish
similar species, and crowd workers learn how to annotate valuable data for
applications like autonomous driving. As humans learn, their brain updates the
visual features it extracts and attend to, which ultimately informs their final
classification decisions. In this work, we propose a novel task of tracing the
evolving classification behavior of human learners as they engage in
challenging visual classification tasks. We propose models that jointly extract
the visual features used by learners as well as predicting the classification
functions they utilize. We collect three challenging new datasets from real
human learners in order to evaluate the performance of different visual
knowledge tracing methods. Our results show that our recurrent models are able
to predict the classification behavior of human learners on three challenging
medical image and species identification tasks.

本文提出了一项新颖的任务，即追踪人类学习者在进行挑战性的视觉分类任务时不断演变的分类行为，提出了一种新的视觉知识追踪模型，并收集三个挑战性的新数据集，从而评估不同的视觉知识追踪方法的性能，结果表明我们的循环模型能够预测人类学习者在三个挑战性的医学图像和物种识别任务中的分类行为。