Rehearsal-based approaches are a mainstay of continual learning (CL). They
mitigate the catastrophic forgetting problem by maintaining a small fixed-size
buffer with a subset of data from past tasks. While most rehearsal-based
approaches study how to effectively exploit the knowledge from the buffered
past data, little attention is paid to the inter-task relationships with the
critical task-specific and task-invariant knowledge. By appropriately
leveraging inter-task relationships, we propose a novel CL method named
DualHSIC to boost the performance of existing rehearsal-based methods in a
simple yet effective way. DualHSIC consists of two complementary components
that stem from the so-called Hilbert Schmidt independence criterion (HSIC):
HSIC-Bottleneck for Rehearsal (HBR) lessens the inter-task interference and
HSIC Alignment (HA) promotes task-invariant knowledge sharing. Extensive
experiments show that DualHSIC can be seamlessly plugged into existing
rehearsal-based methods for consistent performance improvements, and also
outperforms recent state-of-the-art regularization-enhanced rehearsal methods.
Source code will be released.

提出了一种新的基于回顾的连续学习方法 DualHSIC，通过恰当利用任务间关系，从任务特异性和任务不变性的角度提高模型性能，实现了对现有基于回顾的方法的补充和改进。