BriefGPT.xyz
Mar, 2018
随机元优化中短视偏差的理解
Understanding Short-Horizon Bias in Stochastic Meta-Optimization
HTML
PDF
Yuhuai Wu, Mengye Ren, Renjie Liao, Roger Grosse
TL;DR
本文从短时间视角出发,分析元优化算法对学习率设置存在的短视偏差问题,并在标准基准数据集上运行元优化实验,并通过比较不同时间视角下的最优调度进行分析,旨在解决元优化算法在实际神经网络训练过程中表现不佳的问题。
Abstract
Careful tuning of the
learning rate
, or even schedules thereof, can be crucial to effective
neural net training
. There has been much recent interest in gradient-based
→