BriefGPT.xyz
Dec, 2020
课程何时奏效?
When Do Curricula Work?
HTML
PDF
Xiaoxia Wu, Ethan Dyer, Behnam Neyshabur
TL;DR
本文通过实验研究探究按难度排序训练的有效性,发现在标准测试数据集中,curricula只有微弱的优势,证明其优势完全来自动态的训练集大小,同时发现在有限时间预算或数据存在噪声的情况下,curriculum学习可以提高性能,而anti-curriculum则不能。
Abstract
Inspired by human learning, researchers have proposed ordering examples during training based on their difficulty. Both
curriculum learning
, exposing a network to easier examples early in training, and anti-
curriculum l
→