Forgetting refers to the loss or deterioration of previously acquired
information or knowledge. While the existing surveys on forgetting have
primarily focused on continual learning, forgetting is a prevalent phenomenon
observed in various other research domains within deep learning. Forgetting
manifests in research fields such as generative models due to generator shifts,
and federated learning due to heterogeneous data distributions across clients.
Addressing forgetting encompasses several challenges, including balancing the
retention of old task knowledge with fast learning of new tasks, managing task
interference with conflicting goals, and preventing privacy leakage, etc.
Moreover, most existing surveys on continual learning implicitly assume that
forgetting is always harmful. In contrast, our survey argues that forgetting is
a double-edged sword and can be beneficial and desirable in certain cases, such
as privacy-preserving scenarios. By exploring forgetting in a broader context,
we aim to present a more nuanced understanding of this phenomenon and highlight
its potential advantages. Through this comprehensive survey, we aspire to
uncover potential solutions by drawing upon ideas and approaches from various
fields that have dealt with forgetting. By examining forgetting beyond its
conventional boundaries, in future work, we hope to encourage the development
of novel strategies for mitigating, harnessing, or even embracing forgetting in
real applications. A comprehensive list of papers about forgetting in various
research fields is available at
https://github.com/EnnengYang/Awesome-Forgetting-in-Deep-Learning.

遗忘是一个在深度学习中广泛存在的现象，涉及到持续学习、生成模型、联邦学习等领域，在特定情况下遗忘可以具有积极的作用，如隐私保护场景。本综述旨在全面了解遗忘现象，探索不同领域对遗忘的理解，并通过借鉴各领域的思想和方法找到潜在的解决方案。将遗忘研究的边界扩展到更广泛的领域，希望在未来的工作中促进对遗忘的减轻、利用或甚至接纳的新策略的发展。