BriefGPT.xyz
May, 2023
SketchOGD:内存高效的持续学习
SketchOGD: Memory-Efficient Continual Learning
HTML
PDF
Benjamin Wright, Youngjae Min, Jeremy Bernstein, Navid Azizan
TL;DR
本文提出SketchOGD作为解决连续任务中机器学习模型遗忘的内存高效算法,使用在线素描算法压缩模型梯度,并在理论和实验上进行了评估。
Abstract
When
machine learning
models are trained continually on a sequence of tasks, they are liable to forget what they learned on previous tasks -- a phenomenon known as
catastrophic forgetting
. Proposed solutions to <
→