BriefGPT.xyz
Oct, 2023
深入理解抽奖票:抽奖票加速深入理解
Grokking Tickets: Lottery Tickets Accelerate Grokking
HTML
PDF
Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo
TL;DR
通过抽象算法(Lottery Ticket Hypothesis)从完美记忆到完美泛化的过渡阶段,找到网络参数权重的关键性指标,有效描述了学习模式的转变。
Abstract
grokking
is one of the most surprising puzzles in neural network
generalization
: a network first reaches a
memorization
solution with perf
→