Test-time domain adaptation aims to adapt the model trained on source domains
to unseen target domains using a few unlabeled images. Emerging research has
shown that the label and domain information is separately embedded in the
weight matrix and batch normalization (BN) layer. Previous works normally
update the whole network naively without explicitly decoupling the knowledge
between label and domain. As a result, it leads to knowledge interference and
defective distribution adaptation. In this work, we propose to reduce such
learning interference and elevate the domain knowledge learning by only
manipulating the BN layer. However, the normalization step in BN is
intrinsically unstable when the statistics are re-estimated from a few samples.
We find that ambiguities can be greatly reduced when only updating the two
affine parameters in BN while keeping the source domain statistics. To further
enhance the domain knowledge extraction from unlabeled data, we construct an
auxiliary branch with label-independent self-supervised learning (SSL) to
provide supervision. Moreover, we propose a bi-level optimization based on
meta-learning to enforce the alignment of two learning objectives of auxiliary
and main branches. The goal is to use the auxiliary branch to adapt the domain
and benefit main task for subsequent inference. Our method keeps the same
computational cost at inference as the auxiliary branch can be thoroughly
discarded after adaptation. Extensive experiments show that our method
outperforms the prior works on five WILDS real-world domain shift datasets. Our
method can also be integrated with methods with label-dependent optimization to
further push the performance boundary. Our code is available at
this https URL

通过仅操作 BN 层以减少学习干扰和提高域知识学习，结合自我监督学习提供监督，以及使用元学习强制辅助分支与主分支目标对齐的双层优化，我们的方法在五个真实领域转移数据集上优于其他方法。