BriefGPT.xyz
Jul, 2023
神经网络可否进行本地化记忆?
Can Neural Network Memorization Be Localized?
HTML
PDF
Pratyush Maini, Michael C. Mozer, Hanie Sedghi, Zachary C. Lipton, J. Zico Kolter...
TL;DR
通过三种实验方法,我们发现记忆化现象发生在模型的少数层中,而不是最后的几层;其次,我们发现记忆化往往局限于模型的少数神经元或通道;基于这些发现,我们提出了一种新的dropout方法,通过对特定的神经元进行dropout,我们能够减少记忆化的准确性,从而减小泛化差距。
Abstract
Recent efforts at explaining the interplay of
memorization
and
generalization
in deep overparametrized networks have posited that
neural networks
→