We propose a causal framework to explain the catastrophic forgetting in
Class-Incremental Learning (CIL) and then derive a novel distillation method
that is orthogonal to the existing anti-forgetting techniques, such as data
replay and feature/label distillation. We first 1) place CIL into the
framework, 2) answer why the forgetting happens: the causal effect of the old
data is lost in new training, and then 3) explain how the existing techniques
mitigate it: they bring the causal effect back. Based on the framework, we find
that although the feature/label distillation is storage-efficient, its causal
effect is not coherent with the end-to-end feature learning merit, which is
however preserved by data replay. To this end, we propose to distill the
Colliding Effect between the old and the new data, which is fundamentally
equivalent to the causal effect of data replay, but without any cost of replay
storage. Thanks to the causal effect analysis, we can further capture the
Incremental Momentum Effect of the data stream, removing which can help to
retain the old effect overwhelmed by the new data effect, and thus alleviate
the forgetting of the old class in testing. Extensive experiments on three CIL
benchmarks: CIFAR-100, ImageNet-Sub&Full, show that the proposed causal effect
distillation can improve various state-of-the-art CIL methods by a large margin
(0.72%--9.06%).

本文提出了一个因果框架，解释了类增量学习中的灾难性遗忘的原因，并提出了一种新的蒸馏方法，该方法与现有的抗遗忘技术（例如数据回放和特征 / 标签蒸馏）正交。在这个框架下，我们发现特征 / 标签蒸馏虽然存储效率高，但其因果效应与端到端特征学习的优点不相符，而数据回放则能够保留此效应，因此我们建议使用基于 Colliding Effect 的因果蒸馏方法，在不损失回放存储的情况下，实现类增量学习中的遗忘效应的缓解。实验结果表明该方法能够显著提高目前各类增量学习方法的性能。