Continual Test-Time Adaptation (CTTA) aims to adapt a pretrained model to
ever-changing environments during the test time under continuous domain shifts.
Most existing CTTA approaches are based on the Mean Teacher (MT) structure,
which contains a student and a teacher model, where the student is updated
using the pseudo-labels from the teacher model, and the teacher is then updated
by exponential moving average strategy. However, these methods update the MT
model indiscriminately on all parameters of the model. That is, some critical
parameters involving sharing knowledge across different domains may be erased,
intensifying error accumulation and catastrophic forgetting. In this paper, we
introduce Parameter-Selective Mean Teacher (PSMT) method, which is capable of
effectively updating the critical parameters within the MT network under domain
shifts. First, we introduce a selective distillation mechanism in the student
model, which utilizes past knowledge to regularize novel knowledge, thereby
mitigating the impact of error accumulation. Second, to avoid catastrophic
forgetting, in the teacher model, we create a mask through Fisher information
to selectively update parameters via exponential moving average, with
preservation measures applied to crucial parameters. Extensive experimental
results verify that PSMT outperforms state-of-the-art methods across multiple
benchmark datasets. Our code is available at
https://github.com/JiaxuTian/PSMT.

本文引入了 Parameter-Selective Mean Teacher (PSMT) 方法，通过选择性的蒸馏机制在学生模型中利用过去知识对新知识进行正则化，以减轻错误积累的影响；同时，在教师模型中通过 Fisher 信息创建掩码来有选择性地更新参数，并应用保存措施于关键参数，以避免灾难性遗忘，实验证实 PSMT 在多个基准数据集上优于现有方法。