Audio-visual video recognition (AVVR) aims to integrate audio and visual
clues to categorize videos accurately. While existing methods train AVVR models
using provided datasets and achieve satisfactory results, they struggle to
retain historical class knowledge when confronted with new classes in
real-world situations. Currently, there are no dedicated methods for addressing
this problem, so this paper concentrates on exploring Class Incremental
Audio-Visual Video Recognition (CIAVVR). For CIAVVR, since both stored data and
learned model of past classes contain historical knowledge, the core challenge
is how to capture past data knowledge and past model knowledge to prevent
catastrophic forgetting. We introduce Hierarchical Augmentation and
Distillation (HAD), which comprises the Hierarchical Augmentation Module (HAM)
and Hierarchical Distillation Module (HDM) to efficiently utilize the
hierarchical structure of data and models, respectively. Specifically, HAM
implements a novel augmentation strategy, segmental feature augmentation, to
preserve hierarchical model knowledge. Meanwhile, HDM introduces newly designed
hierarchical (video-distribution) logical distillation and hierarchical
(snippet-video) correlative distillation to capture and maintain the
hierarchical intra-sample knowledge of each data and the hierarchical
inter-sample knowledge between data, respectively. Evaluations on four
benchmarks (AVE, AVK-100, AVK-200, and AVK-400) demonstrate that the proposed
HAD effectively captures hierarchical information in both data and models,
resulting in better preservation of historical class knowledge and improved
performance. Furthermore, we provide a theoretical analysis to support the
necessity of the segmental feature augmentation strategy.

音视频识别通过整合音频和视觉线索来准确分类视频。本文旨在解决传统方法在面对真实世界中的新类别时无法保留历史类别知识的问题，提出了一种 Hierarchical Augmentation and Distillation（HAD）方法，通过分层增强以及分层蒸馏模块来高效利用数据和模型的分层结构，以捕捉和保持数据和模型的分层知识，实现更好地保护历史类别知识和提升性能。