BriefGPT.xyz
Mar, 2022
在学习优化器中，内存、计算和性能之间的实际权衡
Practical tradeoffs between memory, compute, and performance in learned optimizers
HTML
PDF
Luke Metz, C. Daniel Freeman, James Harrison, Niru Maheswaranathan, Jascha Sohl-Dickstein
TL;DR
本文主要研究了利用学习优化器代替手动设计的优化器来优化机器学习系统，重点探讨了如何平衡模型训练的内存、计算和性能三方面的权衡，并构建了一个更快捷、更高效的学习优化器。
Abstract
optimization
plays a costly and crucial role in developing
machine learning
systems. In
learned optimizers
, the few
→