Deep neural networks have a clear degradation when applying to the unseen
environment due to the covariate shift. Conventional approaches like domain
adaptation requires the pre-collected target data for iterative training, which
is impractical in real-world applications. In this paper, we propose to adapt
the deep models to the novel environment during inference. An previous solution
is test time normalization, which substitutes the source statistics in BN
layers with the target batch statistics. However, we show that test time
normalization may potentially deteriorate the discriminative structures due to
the mismatch between target batch statistics and source parameters. To this
end, we present a general formulation $\alpha$-BN to calibrate the batch
statistics by mixing up the source and target statistics for both alleviating
the domain shift and preserving the discriminative structures. Based on
$\alpha$-BN, we further present a novel loss function to form a unified test
time adaptation framework Core, which performs the pairwise class correlation
online optimization. Extensive experiments show that our approaches achieve the
state-of-the-art performance on total twelve datasets from three topics,
including model robustness to corruptions, domain generalization on image
classification and semantic segmentation. Particularly, our $\alpha$-BN
improves 28.4\% to 43.9\% on GTA5 $\rightarrow$ Cityscapes without any
training, even outperforms the latest source-free domain adaptation method.

本文提出了一种基于 alpha-BN 的测试时间适应框架 Core，通过混合源和目标统计信息来校准批次统计量以缓解域漂移并保留区分结构。实验证明我们的方法在包括图像分类和语义分割的三个主题的 12 个数据集上实现了最先进的性能。特别地，在没有任何训练的情况下，我们的 alpha-BN 改进了 28.4％至 43.9％，甚至超过了最新的源无关域自适应方法。