BriefGPT.xyz
Nov, 2021
大规模深度学习优化:综述
Large-Scale Deep Learning Optimizations: A Comprehensive Survey
HTML
PDF
Xiaoxin He, Fuzhao Xue, Xiaozhe Ren, Yang You
TL;DR
本文概述了在大规模深度学习中如何优化模型的准确性和效率,讨论了优化中使用的算法、大批量训练中出现的泛化差距问题,并回顾了最新的解决通信负担和减少内存占用的策略。
Abstract
deep learning
have achieved promising results on a wide spectrum of AI applications. Larger datasets and models consistently yield better performance. However, we generally spend longer training time on more computation and
→