Test-time adaptation (TTA) is a task that continually adapts a pre-trained source model to the target domain during inference. One popular approach involves fine-tuning model with cross-entropy loss according to estimated pseudo-labels. However, its performance is significantly affected by noisy pseudo-labels. This study reveals that minimizing the classification error of each sample causes the cross-entropy loss's vulnerability to label noise. To address this issue, we propose a novel Decoupled Prototype Learning (DPL) method that features prototype-centric loss computation. First, we decouple the optimization of class prototypes. For each class prototype, we reduce its distance with positive samples and enlarge its distance with negative samples in a contrastive manner. This strategy prevents the model from overfitting to noisy pseudo-labels. Second, we propose a memory-based strategy to enhance DPL's robustness for the small batch sizes often encountered in TTA. We update each class's pseudo-feature from a memory in a momentum manner and insert an additional DPL loss. Finally, we introduce a consistency regularization-based approach to leverage samples with unconfident pseudo-labels. This approach transfers feature styles of samples with unconfident pseudo-labels to those with confident pseudo-labels. Thus, more reliable samples for TTA are created. The experimental results demonstrate that our methods achieve state-of-the-art performance on domain generalization benchmarks, and reliably improve the performance of self-training-based methods on image corruption benchmarks. The code will be released.

通过提出Decoupled Prototype Learning (DPL)方法，本研究解决了使用伪标签进行交叉熵损失微调时容易受到标签噪声影响的问题，并引入了基于内存的策略增强小批量处理的鲁棒性，在遇到不确定伪标签的样本时，使用一致性正则化方法转移特征风格以提高测试时自适应（TTA）的可靠性。实验结果表明，我们的方法在领域泛化基准上取得了最先进的性能，并可靠提高图像破坏基准上基于自训练方法的性能。

可靠的测试时间适应性的解耦原型学习