BriefGPT.xyz
Feb, 2019
深度神经网络中稀疏性的状态
The State of Sparsity in Deep Neural Networks
HTML
PDF
Trevor Gale, Erich Elsen, Sara Hooker
TL;DR
本文评估了三种在深度神经网络中引入稀疏性的技术,并对两个大规模的学习任务进行了严格评估,结果表明,简单的幅度剪枝方法可以获得相当或更好的性能,而不能从头开始训练稀疏结构,并强调了建立大规模基准测试的必要性。
Abstract
We rigorously evaluate three state-of-the-art techniques for inducing
sparsity
in
deep neural networks
on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained o
→