Gradient preconditioning is a key technique to integrate the second-order
information into gradients for improving and extending gradient-based learning
algorithms. In deep learning, stochasticity, nonconvexity, and high
dimensionality lead to a wide variety of gradient preconditioning methods, with
implementation complexity and inconsistent performance and feasibility. We
propose the Automatic Second-order Differentiation Library (ASDL), an extension
library for PyTorch, which offers various implementations and a plug-and-play
unified interface for gradient preconditioning. ASDL enables the study and
structured comparison of a range of gradient preconditioning methods.

本文提出基于 Automatic Second-order Differentiation Library (ASDL) for PyTorch 的梯度预处理方法，该方法可以以一种统一的方式，实现和比较多种梯度预处理方法。

ASDL：PyTorch 梯度预调接口统一化

ASDL: A Unified Interface for Gradient Preconditioning in PyTorch

In a real-world setting, object instances from new classes can be
continuously encountered by object detectors. When existing object detectors
are applied to such scenarios, their performance on old classes deteriorates
significantly. A few efforts have been reported to address this limitation, all
of which apply variants of knowledge distillation to avoid catastrophic
forgetting. We note that although distillation helps to retain previous
learning, it obstructs fast adaptability to new tasks, which is a critical
requirement for incremental learning. In this pursuit, we propose a
meta-learning approach that learns to reshape model gradients, such that
information across incremental tasks is optimally shared. This ensures a
seamless information transfer via a meta-learned gradient preconditioning that
minimizes forgetting and maximizes knowledge transfer. In comparison to
existing meta-learning methods, our approach is task-agnostic, allows
incremental addition of new-classes and scales to high-capacity models for
object detection. We evaluate our approach on a variety of incremental learning
settings defined on PASCAL-VOC and MS COCO datasets, where our approach
performs favourably well against state-of-the-art methods.

该研究提出了一种元学习方法，以优化共享增量任务信息的模型梯度并通过元学习梯度预处理来减少遗忘和最大化知识转移，从而提高目标检测中的增量学习性能。