BriefGPT.xyz
Dec, 2018
深度神经网络学习中示例遗忘的实证研究
An Empirical Study of Example Forgetting during Deep Neural Network Learning
HTML
PDF
Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio...
TL;DR
研究神经网络在单分类任务训练中的学习动态，发现在缺乏明显分布偏移的数据情况下，存在相关遗忘现象，某些样例更容易被遗忘，而基于遗忘动态可以从训练数据集中省略部分例子却仍能保持最佳泛化性能。
Abstract
Inspired by the phenomenon of
catastrophic forgetting
, we investigate the learning dynamics of
neural networks
as they train on single
classifica
→