BriefGPT.xyz
Dec, 2023
理解和利用神经网络的学习阶段
Understanding and Leveraging the Learning Phases of Neural Networks
HTML
PDF
Johannes Schneider, Mohit Prabhushanka
TL;DR
通过对参数的演化,我们全面分析了深度神经网络的学习动态,发现存在三个阶段:接近恒定的重建损失、下降和上升。我们还通过经验实证建立了数据模型,并对单层神经网络证明了阶段的存在。我们的工作为迁移学习提供了新的最佳实践:通过实验证明预训练的分类器在性能达到最优之前应该停止。
Abstract
The
learning dynamics
of
deep neural networks
are not well understood. The information bottleneck (IB) theory proclaimed separate fitting and compression phases. But they have since been heavily debated. We compr
→