Grokking, or delayed generalization, is a phenomenon where generalization in
a deep neural network (DNN) occurs long after achieving near zero training
error. Previous studies have reported the occurrence of grokking in specific
controlled settings, such as DNNs initialized with large-norm parameters or
transformers trained on algorithmic datasets. We demonstrate that grokking is
actually much more widespread and materializes in a wide range of practical
settings, such as training of a convolutional neural network (CNN) on CIFAR10
or a Resnet on Imagenette. We introduce the new concept of delayed robustness,
whereby a DNN groks adversarial examples and becomes robust, long after
interpolation and/or generalization. We develop an analytical explanation for
the emergence of both delayed generalization and delayed robustness based on a
new measure of the local complexity of a DNN's input-output mapping. Our local
complexity measures the density of the so-called 'linear regions' (aka, spline
partition regions) that tile the DNN input space, and serves as a utile
progress measure for training. We provide the first evidence that for
classification problems, the linear regions undergo a phase transition during
training whereafter they migrate away from the training samples (making the DNN
mapping smoother there) and towards the decision boundary (making the DNN
mapping less smooth there). Grokking occurs post phase transition as a robust
partition of the input space emerges thanks to the linearization of the DNN
mapping around the training points. Website: this https URL

深度神经网络的一项研究探索了 “Grokking” 或延迟泛化的现象及其对网络鲁棒性和局部复杂性的影响。

深度网络始终能理解，理由如下

Deep Networks Always Grok and Here is Why

The study of Deep Network (DN) training dynamics has largely focused on the
evolution of the loss function, evaluated on or around train and test set data
points. In fact, many DN phenomenon were first introduced in literature with
that respect, e.g., double descent, grokking. In this study, we look at the
training dynamics of the input space partition or linear regions formed by
continuous piecewise affine DNs, e.g., networks with (leaky)ReLU
nonlinearities. First, we present a novel statistic that encompasses the local
complexity (LC) of the DN based on the concentration of linear regions inside
arbitrary dimensional neighborhoods around data points. We observe that during
training, the LC around data points undergoes a number of phases, starting with
a decreasing trend after initialization, followed by an ascent and ending with
a final descending trend. Using exact visualization methods, we come across the
perplexing observation that during the final LC descent phase of training,
linear regions migrate away from training and test samples towards the decision
boundary, making the DN input-output nearly linear everywhere else. We also
observe that the different LC phases are closely related to the memorization
and generalization performance of the DN, especially during grokking.

在这项研究中，我们研究了连续分段仿射深度网络（如带有（渗漏的）ReLU 非线性的网络）训练动态的输入空间划分或线性区域，提出了一种包含了本地复杂性的新统计量，观察到在训练过程中，数据点周围的本地复杂性经历了几个阶段，与深度网络的记忆性和概括性能密切相关。