Test-time domain adaptation aims to adapt the model trained on source domains
to unseen target domains using a few unlabeled images. Emerging research has
shown that the label and domain information is separately embedded in the
weight matrix and batch normalization (BN) layer. Previous works normally
update the whole network naively without explicitly decoupling the knowledge
between label and domain. As a result, it leads to knowledge interference and
defective distribution adaptation. In this work, we propose to reduce such
learning interference and elevate the domain knowledge learning by only
manipulating the BN layer. However, the normalization step in BN is
intrinsically unstable when the statistics are re-estimated from a few samples.
We find that ambiguities can be greatly reduced when only updating the two
affine parameters in BN while keeping the source domain statistics. To further
enhance the domain knowledge extraction from unlabeled data, we construct an
auxiliary branch with label-independent self-supervised learning (SSL) to
provide supervision. Moreover, we propose a bi-level optimization based on
meta-learning to enforce the alignment of two learning objectives of auxiliary
and main branches. The goal is to use the auxiliary branch to adapt the domain
and benefit main task for subsequent inference. Our method keeps the same
computational cost at inference as the auxiliary branch can be thoroughly
discarded after adaptation. Extensive experiments show that our method
outperforms the prior works on five WILDS real-world domain shift datasets. Our
method can also be integrated with methods with label-dependent optimization to
further push the performance boundary. Our code is available at
this https URL

通过仅操作 BN 层以减少学习干扰和提高域知识学习，结合自我监督学习提供监督，以及使用元学习强制辅助分支与主分支目标对齐的双层优化，我们的方法在五个真实领域转移数据集上优于其他方法。

通过学习领域感知的批归一化实现测试时域自适应

Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization

Pre-trained models have achieved excellent performance on the dialogue task.
However, for the continual increase of online chit-chat scenarios, directly
fine-tuning these models for each of the new tasks not only explodes the
capacity of the dialogue system on the embedded devices but also causes
knowledge forgetting on pre-trained models and knowledge interference among
diverse dialogue tasks. In this work, we propose a hierarchical inductive
transfer framework to learn and deploy the dialogue skills continually and
efficiently. First, we introduce the adapter module into pre-trained models for
learning new dialogue tasks. As the only trainable module, it is beneficial for
the dialogue system on the embedded devices to acquire new dialogue skills with
negligible additional parameters. Then, for alleviating knowledge interference
between tasks yet benefiting the regularization between them, we further design
hierarchical inductive transfer that enables new tasks to use general knowledge
in the base adapter without being misled by diverse knowledge in task-specific
adapters. Empirical evaluation and analysis indicate that our framework obtains
comparable performance under deployment-friendly model capacity.

本文提出了一种基于递归传递的框架，通过引入适配器模块和迁移学习来解决在线聊天场景中为每个新任务调整模型导致容量过大及知识忘却问题。实验结果表明，在部署友好型模型容量下，我们的框架表现出可比较的性能