Fine-tuning pre-trained neural network models has become a widely adopted
approach across various domains. However, it can lead to the distortion of
pre-trained feature extractors that already possess strong generalization
capabilities. Mitigating feature distortion during adaptation to new target
domains is crucial. Recent studies have shown promising results in handling
feature distortion by aligning the head layer on in-distribution datasets
before performing fine-tuning. Nonetheless, a significant limitation arises
from the treatment of batch normalization layers during fine-tuning, leading to
suboptimal performance. In this paper, we propose Domain-Aware Fine-Tuning
(DAFT), a novel approach that incorporates batch normalization conversion and
the integration of linear probing and fine-tuning. Our batch normalization
conversion method effectively mitigates feature distortion by reducing
modifications to the neural network during fine-tuning. Additionally, we
introduce the integration of linear probing and fine-tuning to optimize the
head layer with gradual adaptation of the feature extractor. By leveraging
batch normalization layers and integrating linear probing and fine-tuning, our
DAFT significantly mitigates feature distortion and achieves improved model
performance on both in-distribution and out-of-distribution datasets. Extensive
experiments demonstrate that our method outperforms other baseline methods,
demonstrating its effectiveness in not only improving performance but also
mitigating feature distortion.

本文介绍了 Domain-Aware Fine-Tuning (DAFT) 的方法，该方法通过批量归一化转换和线性探测与微调的集成来有效减轻特征扭曲，并在分布和非分布数据集上实现了改进的模型性能。